PROJECT MANAGEMENT
...
Annotation Requests
Request details

Summary

6min

Within the Summary tab, you find information with the intention to help you get a good overview of the request and its current progress.

Input Progress

In this card, you can monitor the progress of the request on an input level. The information is intended to help you answer questions such as:

  • In which state are the majority of the inputs?
  • Are there any delivery-ready annotations that I could download already?
  • Do we need to review more data?

An input is a set of sensor data that you want to be annotated as a single entity. It usually consists of at least one image/frame. It can also contain images from several cameras, a point cloud, a video or a sequence of images. The input is a part of an input batch i.e. a dataset.

ο»Ώ

The input progress card
The input progress card
ο»Ώ

Inputs with delivery-ready annotations

Shows the percentage and the absolute number of inputs that have a delivery-ready annotation.

Workflow Stage Distribution

This bar chart shows the distribution of inputs between the request's different workflow stages. On the input's way toward getting a delivery-ready annotation, it goes through a series of workflow stages. Each stage corresponds to one action in one of the workflow's different phases, such as the action β€œCorrect” in the phase β€œReview”.

Which stages the input goes through depends on the workflow used in the request. An input can only be in one workflow stage at a time.

Generally, inputs "move forward" in the workflow except that they can circle between the actions inside a phase. So, for example, an input can move in between the actions Review and Correct in the Review phase before moving forward to the next phase.

Which stages do I see?

Which stages the input goes through depends on the workflow used in the request. Some stages are only visible to the organization being Producer for the request.

An input can only be in one stage at a time. You can read more about the stages below.

What do the workflow stages mean?

Production

  • Not started Inputs that are waiting to be annotated.
  • Annotation: Annotate* Inputs that are being annotated using annotate tasks.
  • Sampled QA: Quality Assurance* Annotated inputs waiting to be or currently are being quality assured as a part of the annotation phase CQC. It also includes inputs that are waiting for a quality decision based on the outcome of sampled quality assurance.
  • Correction: Correct* Annotated inputs waiting to be or currently being quality assured using correction tasks.
  • Review: Review Annotated inputs that are waiting to be or currently are being reviewed.
  • Review: Correct Annotated inputs that got rejected in review and now either are waiting to be or currently are being corrected.

Delivery

  • Delivery: Ready Annotated inputs that have gotten their final annotations and thus are ready for delivery.

*The stages Quality Assurance and Correction are only shown for organizations with the request role Producer. For organizations with other request roles, these stages are part of the stage In Annotation.

ο»Ώ

Updated 11 Jul 2023
Doc contributor
Doc contributor
Doc contributor
Did this page help you?