Settings
Request Settings allows you to manage and customize how annotation requests behave within the Kognic platform. You can access these settings from the Project Management area by navigating to a specific request and selecting the settings option.
The settings are organized into two tabs: General and Phases.
General tab
The General tab contains core configuration options for your request, organized into several sections.
Request name
The request name functions as an identifier in multiple locations. Ensure you choose a name that makes the request easy to identify. The name cannot be emptyβwhitespace-only values will be trimmed.
Update the request name if its purpose has changed or to improve clarity for your team.
Request workflow
Only available for non-legacy workflows.
Change the workflow template used for this request. The workflow controls which phases and workflow stages are available.
You can change the workflow if you are an Admin or Project Manager and your organization both owns the workspace and is a Producer on the request. That is, teams annotating in their own workspace. If Kognic or a partner produces your annotations, workflow changes are handled by Kognic.
You can only modify this setting before the request has started processing inputs. Use this if requirements change before production begins and you need to switch to a different workflow template.
Kognic-managed and custom workflows
If the request's current workflow is not one of the workflows available to your organization, the field shows a read-only value instead of a selection:
- Kognic-managed workflow - the workflow was set up by Kognic and is managed by Kognic. It is not available in the workflow list.
- Custom workflow - the phases of this request were tailored and do not match any standard workflow.
Select Change to switch to another workflow. The list opens with the current workflow pre-selected and marked Current. Leaving it selected keeps the current workflow and stages no change. You can switch away from a Kognic-managed workflow, but only Kognic can switch it back.
Organizational request roles
These roles define how organizations collaborate on the request and determine access levels and responsibilities.
Request owner
The organization that houses the project and request within the platform. The owner:
- Uploads the data to be annotated
- Is the end consumer of the processed/annotated information
- Once production is complete, the request owner can quality assure annotations either within the request workflow or during the Delivery phase. Within the workflow, quality assurance happens in dedicated Client QA or sampled review phases. You can enable this under Request settings/Phases, if the workflow supports it.
- Users of the owner organization have limited monitoring and management capabilities in the request's production phases.
This field is read-only.
ο»ΏRequest producerο»Ώ
The organization responsible for producing annotated data in the request. The producer:
- Manages the annotation production process
- Has access to detailed monitoring and management options
- Can configure the request workforce in detail
Users from the producer organization have expanded permissions compared to non-producer organizations.
Client QA Contributor
Organizations specified as Client QA contributors support the owner organization with conducting quality assurance of produced annotations.
After production is complete, Client QA contributors can quality assure annotations in two ways: within the request workflow or during the Delivery phase. Within the workflow, quality assurance occurs in dedicated Client QA or sampled review phases. You can enable this under Request settings/Phases if the workflow supports it.
Input batch information
Only available to Kognic users.
This section displays the input batch associated with the request. An input batch contains the scenes and inputs that will be labeled.
β οΈ Important: The input batch can be used in multiple requests. Any changes made here will affect the batch globally across all requests using it.
Field | Description | Editable |
|---|---|---|
External ID | Customer-defined identifier for the batch. Useful for mapping to external systems. | Yes |
Batch name | Human-readable name for the batch. | Yes |
Click the Edit button next to each field to open an edit dialog. Changes are saved immediately upon confirmation.
When editing, keep in mind:
- External ID cannot be empty
- External ID must be unique within the project
- Batch name cannot be empty
Annotation instruction
Configure what to annotate and how annotations should be performed. This section displays differently depending on whether your request uses the legacy or modern instruction format.
Modern annotation instructions
For requests using the current instruction format, you can configure:
Instruction β Select which Annotation Instruction to use for this request. An Annotation Instruction specifies what you want to annotate and provides guidance on how to perform the annotation.
Revision β Select which revision (version) of the instruction to use. A revision represents a specific version of the instruction that has been used in production. As you learn more about your dataset, you can create new revisions to update guidelines.
Warnings when changing instructions
Warning | When it appears | What it means |
|---|---|---|
Multi-sensor scenes mismatch | Instruction configured for single lidar but request has multi-sensor scenes (or vice versa) | May cause annotation issues |
Pre-annotation warning | Request contains pre-annotations | Instruction must match pre-annotation format |
Corrections may be needed | Ongoing or completed inputs exist | Switching revisions may require correction tasks |
Multiple active instructions | Switching causes multiple revisions to be active in the project | Custom checkers and monitoring may behave unexpectedly |
Changing annotation instruction revision in a way that introduce breaking changes is only allowed if the annotation request is not automatically allocating tasks, and if there are no completed tasks. If you experience issues in trying to make such an update, make sure the request adhere to these requirements before contacting support.
Legacy task definition (deprecated)
For requests using the deprecated instruction format, the Task Definition name and Revision are displayed as read-only fields. A deprecation notification recommends migrating to the new Annotation Instructions format. These settings cannot be edited within Request Settings.
Phases tab
The Phases tab allows you to configure workflow phases for non-legacy requests. Only phases with configurable settings are displayed.
β οΈ Important: All changes to configurations will impact future outcomes exclusively. Changes will not retroactively affect previously processed inputs.
General settings
Workflow checkpoints
Only available to Kognic users, and only for non-legacy workflows.
Enable more specific progress tracking for request inputs by defining workflow checkpoints. This is an experimental feature.
Production complete checkpoint β Defines when an input is considered to have completed the production flow.
Setting | Description |
|---|---|
Checkpoint placement | Select which workflow phase the checkpoint should be placed before |
Effect | When an input passes through this checkpoint, it becomes available to the client for quality assurance |
Default | If not defined, the checkpoint is placed before the Delivery phase |
β οΈ Important: Once saved, the checkpoint placement cannot be edited. A critical warning is displayed before saving, and after saving, the field becomes read-only with a locked indicator.
Phase settings
Each workflow phase has specific configurable options. The available settings depend on the phase type.
Common settings
These settings appear across multiple phase types:
Setting | Description | Available in |
|---|---|---|
Keep reviewer throughout phase | Follow-up reviews are automatically assigned to the person who originally reviewed the input in this phase | Sampled Review, Full Review, Client QA, Delivery |
Assign corrections to last input assignee | When a review is rejected, the correction task is assigned to the team member who submitted the last task before the review, regardless of phase | Sampled Review, Full Review, Client QA, Delivery |
Allow team member to perform consecutive tasks | Allows the same person to perform consecutive tasks. Primarily used for testing purposes | All phases |
Focused review | For corrections: tasks include a filtered list of objects that received feedback. For review rounds 2+: tasks include objects that addressed feedback and new objects created in correction | Sampled Review, User Quality Review, Expert Verification, Client QA |
Sampled Review phase
A review phase where only a percentage of inputs are selected for review.
Sampling rate β Set the probability (0-100%) of an input entering the phase being selected for review. Changes only affect inputs that have not yet entered the phase.
Additional settings available:
Setting | Description |
|---|---|
Keep reviewer throughout phase | Follow-up reviews assigned to original reviewer |
Assign corrections to last input assignee | Corrections go to last task submitter |
Allow team member to perform consecutive tasks | Same person can do sequential tasks |
Allow workspace owner to manage review stage | Grants owner permission to manage review actions |
Focused review | Filters objects in correction and follow-up reviews |
Prevent editing of annotations in review tasks | Annotations can be viewed but not edited in review. Feedback can still be added and edited. This applies to all ongoing review tasks. |
Full Review phase
A review phase where all inputs are reviewed.
Setting | Description |
|---|---|
Allow team member to perform consecutive tasks | Same person can do sequential tasks |
Keep reviewer throughout phase | Follow-up reviews assigned to original reviewer |
Assign corrections to last input assignee | Corrections go to last task submitter |
Quality Review phase
A quality assurance review phase for monitoring annotator performance.
Setting | Description |
|---|---|
Focused review | Filters objects in correction and follow-up reviews |
Expert Verification phase
A verification phase for expert-level quality checks.
Setting | Description |
|---|---|
Focused review | Filters objects in correction and follow-up reviews |
Client QA Review phase
A quality assurance phase for client review of annotations.
Sampling rate β Set the probability (0-100%) of an input being selected for client QA. Changes only affect inputs that have not yet entered the phase.
Additional settings available:
Setting | Description |
|---|---|
Keep reviewer throughout phase | Follow-up reviews assigned to original reviewer |
Assign corrections to last input assignee | Corrections go to last task submitter |
Allow team member to perform consecutive tasks | Same person can do sequential tasks |
Focused review | Filters objects in correction and follow-up reviews |
Correction phase
A phase where annotators address feedback from review stages.
Setting | Description |
|---|---|
Allow team member to perform consecutive tasks | Same person can do sequential tasks |
Delivery phase
The final phase, where completed annotations are delivered.
Setting | Description |
|---|---|
Allow team member to perform consecutive tasks | Same person can do sequential tasks |
Keep reviewer throughout phase | Follow-up reviews assigned to original reviewer |
Assign corrections to last input assignee | Corrections go to last task submitter |
Setting | Description | Available in |
|---|---|---|
Keep reviewer throughout phase | Follow-up reviews are automatically assigned to the person who originally reviewed the input in this phase | Sampled Review, Full Review, Client QA, Delivery |
Assign corrections to last input assignee | When a review is rejected, the correction task is assigned to the team member who submitted the last task before the review, regardless of phase | Sampled Review, Full Review, Client QA, Delivery |
Allow team member to perform consecutive tasks | Allows the same person to perform consecutive tasks. Primarily used for testing purposes | All phases |
Focused review | For corrections: tasks include a filtered list of objects that received feedback. For review rounds 2+: tasks include objects that addressed feedback and new objects created in correction | Sampled Review, User Quality Review, Expert Verification, Client QA |