If a reviewer rejects the task you (or a fellow annotator) submitted, you will receive it as a correction task. This correction task differs slightly from regular correction tasks in that it includes feedback.

In the previous section, Read feedback you can read about how to find feedback and use the feedback list and cards.

In this section, we describe how to tackle the feedback that you got! 💪

When you receive correction requests, the reviewer expects you to fix what they have given you feedback on. The reviewer is instructed to provide constructive and clear feedback, but unclarities could remain.

Fix the error according to the feedback and then resolve the correction request. You resolve by clicking the resolve button on the feedback item (it is not done automatically).

Keep an eye out for errors that exist in multiple frames The reviewer can't indicate that an error occurs in multiple frames using our feedback tools. You have to double-check if the identified error occurs in other frames. For example, when the reviewer has marked out that an object is missing in the frame, you have to double-check if it exists in other frames of the sequence. If it is stated clearly by the reviewer that the error only exists in one frame. You don't have to double-check the other frames.

If you feel that you made the correct decision when annotating and don't agree with the reviewer's suggested fix, you should ask for clarification and use it as a learning opportunity for both you and the reviewer.

So, don't resolve the correction request; write a reply in which you explain your reasoning instead.

If you don't understand the correction request, don't resolve it. Write a reply saying that you don't understand and need clarification.

If you believe the feedback is invalid, mark it as invalid. For example, the feedback lists a missing object very far away, but the guideline states to mark only nearby objects.

The goal with advice is to give you, as an annotator, general tips on how to improve or encourage words.

How you act on these is fully up to you; you can resolve and/or reply if you want - but you don't have to.