PROJECT MANAGEMENT

Annotation Workflows

7min

In the Kognic platform, annotations can be produced using a selection of workflows. These workflows determine the steps used to produce the annotations and in what order the steps should happen. They define what type of steps (tasks) to complete before we have produced a deliverable annotation.

Multiple factors, such as the desired quality, time, and cost, determine which workflow suits each request.

Our available workflows

πŸ€Έβ€β™€οΈ Flexible

  • Annotation + Full review + Sampled review
  • Annotation + Sampled review + Sampled review

Read about them in detail on the page Flexible Workflowsο»Ώ

🎩 Classic

  • Annotate + Correct + Review
  • Annotate + Review

Read about them in detail on the page Classic Workflowsο»Ώ

Select a request workflow

When creating a new request, you have to select which workflow to use. The workflows available for selection are either classic or flexible. Our classic workflows are designed for stability, efficiency, and familiarity, while our flexible workflows provide the same benefits along with increased flexibility.

You can read more about creating requests and selecting their workflow in the guides: Guide: Create a requestο»Ώ and Guide: Create a new projectο»Ώ.

ο»Ώ

Key concepts

Task

A task is what ties an input to a team member and what type of action the team member should take with the input. There are three types of tasks - annotate, correct, and review. On its way from original input to a delivery-ready (or final) annotation, the input spawns one or multiple tasks, sometimes even multiple tasks of the same type. The number of tasks depends on the workflows used in the request and if tasks expire and get assigned to new users.

Task types

Annotate An annotate task will be given to a team member with the purpose of them annotating it from scratch. It is always the initial task inside the annotation workflow. Correct A task where the team member is asked to correct a task with existing annotations. Depending on the workflow, a Correct task follows either each Annotate task or tasks that got rejected during a Review task. Review Review tasks are similar to Correct tasks but allow a team member to give structured feedback on objects, specific timestamps, and specific sensors inside the app. Review tasks also include a final judgement on whether to Accept or Reject the reviewed annotation

Task Assignments

For a user to be able to work on a task it has to be assigned to the user. The user can now complete the assignment or let it expire. Once it expires, the task becomes available for assignment to other users.

The task is assigned either automatically by our system or manually by a manager, see Teamο»Ώ for more details.

ο»Ώ

ο»Ώ