Introduction

🕵🏻‍♀️ as a person performing reviews, we suggest you read the pages add feedback docid\ dyap cmm72dmzpvz6e5lb , accept/ reject tasks docid\ agjc0n wq1jrl9i2moyk and read feedback docid\ w0xvei0tfpxzaj8u9msx4 👨🏼‍🎨 as an annotator, we suggest you read the pages read feedback docid\ w0xvei0tfpxzaj8u9msx4 and act on the feedback docid\ a3q9wn exy2 wqsohtwfo feedback inside the review phase in app feedback aims to enable clear and efficient communication between the reviewing person (giving feedback) and the person who did the initial task, respectively, who receives the reviewed task (receiving feedback, acting on it, responding to it) feedback passed around outside the app via slack, teams, email, or spreadsheets is slower to act on and prone to errors advantages of the review phase give feedback directly inside the app rather than via spreadsheets or other ways give feedback with direct reference to the annotated object in an image or a location inside a point cloud give feedback inside the normal workflow with less delay after task completion make small corrections inside the tool to avoid another iteration for small errors how it works if enabled, the "review phase" starts once a preliminary annotation exists, and its flow is as follows (an annotator annotated a task and potentially already did some corrections) a reviewer reviews the task potentially adds feedback and/or correction requests and most importantly decides whether to accept or reject the task as a good final annotation in accordance with the guideline an accepted task is done and soon ready to download by the client in the future annotator s will also receive feedback from accept ed tasks, today that's not supported yet a rejected annotation is assigned to an annotator, if possible the original one, to correct the correction requests and acknowledge or act on the feedback reply to both of the above, if needed submits the corrected annotation a reviewer reviews again and she still reject s step 4 is repeated until she accept s correction request vs feedback in our review phase we use two types of feedback correction requests and advice correction requests a correction request represents something the person reviewing a task want to have corrected this means if an annotator recieves a correction task with correction requests in it she is expected to correct the error the reviewer have indicated advice these items represents something the person reviewing wants the annotator to know or think about, but not something they expect you to correct advice could be things such as encouraging words and general tips on how to improve feedback conversations discussing on the right way to do annotations can quickly become complex and details start to matter your feedback conversations might be much better if you consider the following tipps giving feedback concentrate on the annotation content, not the person for example, "you forgot to add a box here" vs "i think that according to the guidelines there should be a box for this parked car" also give positive, encouraging feedback try to be specific for example "behind the trees there's some parking cars missing" vs "to me it seems there's five parking cars behind the trees" feedback is your personal opinion and might be debateable, phrase it as such receiving feedback be aware of responses be open to different opinions ask for clarifications if you don't understand how to act on feedback