In the Kognic platform, annotations can be produced using a selection of workflows. These workflows determine the steps used to produce the annotations and in what order the steps should happen. They define what type of steps (tasks) to complete before we have produced a deliverable annotation.

Multiple factors, such as the desired quality, time, and cost, determine which workflow suits each request.

Phases: Annotate -> Full review --> Sampled review -> Delivery Availability: All organizations

This request workflow features two quality assurance phases. In the initial review phase, all inputs are selected for review and require reviewer acceptance to complete the phase. The second review examines only a sample of inputs. Insights from this sample help estimate the quality of the remaining annotated inputs. This approach allows for reduced review effort when the estimated quality is good, while still enabling full review when needed.

Read more in Two-tier review.

Phases: Annotate -> Sampled review --> Sampled review -> Delivery Availability: All organizations

This request workflow features two sampled review phases. In these phases, only a subset of inputs is initially selected for review. Insights from this sample help estimate the quality of the remaining annotated inputs. This approach allows for reduced review effort when the estimated quality is good, while still enabling full review when needed.

Read more in Two-tier sampled review.

A task is what ties an input to a team member and what type of action the team member should take with the input. There are three types of tasks - annotate, correct, and review. On its way from original input to a delivery-ready (or final) annotation, the input spawns one or multiple tasks, sometimes even multiple tasks of the same type. The number of tasks depends on the workflows used in the request and if tasks expire and get assigned to new users.

Annotate An annotate task will be given to a team member with the purpose of them annotating it from scratch. It is always the initial task inside the annotation workflow. Correct A task where the team member is asked to correct a task with existing annotations. Depending on the workflow, a Correct task follows either each Annotate task or tasks that got rejected during a Review task. Review Review tasks are similar to Correct tasks but allow a team member to give structured feedback on objects, specific timestamps, and specific sensors inside the app. Review tasks also include a final judgement on whether to Accept or Reject the reviewed annotation

For a user to be able to work on a task it has to be assigned to the user. The user can now complete the assignment or let it expire. Once it expires, the task becomes available for assignment to other users.

The task is assigned either automatically by our system or manually by a manager, see Workforce for more details.