Settings

request settings allows you to manage and customize how annotation requests behave within the kognic platform you can access these settings from the project management area by navigating to a specific request and selecting the settings option the settings are organized into two tabs general and phases general tab the general tab contains core configuration options for your request, organized into several sections request name the request name functions as an identifier in multiple locations ensure you choose a name that makes the request easy to identify the name cannot be empty—whitespace only values will be trimmed update the request name if its purpose has changed or to improve clarity for your team request workflow only available for non legacy workflows change the workflow template used for this request the workflow controls which phases and workflow stages are available you can change the workflow if you are an admin or project manager and your organization both owns the workspace and is a producer on the request that is, teams annotating in their own workspace if kognic or a partner produces your annotations, workflow changes are handled by kognic you can only modify this setting before the request has started processing inputs use this if requirements change before production begins and you need to switch to a different workflow template kognic managed and custom workflows if the request's current workflow is not one of the workflows available to your organization, the field shows a read only value instead of a selection kognic managed workflow the workflow was set up by kognic and is managed by kognic it is not available in the workflow list custom workflow the phases of this request were tailored and do not match any standard workflow select change to switch to another workflow the list opens with the current workflow pre selected and marked current leaving it selected keeps the current workflow and stages no change you can switch away from a kognic managed workflow, but only kognic can switch it back organizational request roles these roles define how organizations collaborate on the request and determine access levels and responsibilities request owner the organization that houses the project and request within the platform the owner uploads the data to be annotated is the end consumer of the processed/annotated information once production is complete, the request owner can quality assure annotations either within the request workflow or during the delivery phase within the workflow, quality assurance happens in dedicated client qa or sampled review phases you can enable this under request settings/phases, if the workflow supports it users of the owner organization have limited monitoring and management capabilities in the request's production phases this field is read only request producer the organization responsible for producing annotated data in the request the producer manages the annotation production process has access to detailed monitoring and management options can configure the request workforce in detail users from the producer organization have expanded permissions compared to non producer organizations client qa contributor organizations specified as client qa contributors support the owner organization with conducting quality assurance of produced annotations after production is complete, client qa contributors can quality assure annotations in two ways within the request workflow or during the delivery phase within the workflow, quality assurance occurs in dedicated client qa or sampled review phases you can enable this under request settings/phases if the workflow supports it input batch information only available to kognic users this section displays the input batch associated with the request an input batch contains the scenes and inputs that will be labeled ⚠️ important the input batch can be used in multiple requests any changes made here will affect the batch globally across all requests using it field description editable external id customer defined identifier for the batch useful for mapping to external systems yes batch name human readable name for the batch yes click the edit button next to each field to open an edit dialog changes are saved immediately upon confirmation when editing, keep in mind external id cannot be empty external id must be unique within the project batch name cannot be empty annotation instruction configure what to annotate and how annotations should be performed this section displays differently depending on whether your request uses the legacy or modern instruction format modern annotation instructions for requests using the current instruction format, you can configure instruction — select which annotation instruction to use for this request an annotation instruction specifies what you want to annotate and provides guidance on how to perform the annotation revision — select which revision (version) of the instruction to use a revision represents a specific version of the instruction that has been used in production as you learn more about your dataset, you can create new revisions to update guidelines warnings when changing instructions warning when it appears what it means multi sensor scenes mismatch instruction configured for single lidar but request has multi sensor scenes (or vice versa) may cause annotation issues pre annotation warning request contains pre annotations instruction must match pre annotation format corrections may be needed ongoing or completed inputs exist switching revisions may require correction tasks multiple active instructions switching causes multiple revisions to be active in the project custom checkers and monitoring may behave unexpectedly changing annotation instruction revision in a way that introduce breaking changes is only allowed if the annotation request is not automatically allocating tasks, and if there are no completed tasks if you experience issues in trying to make such an update, make sure the request adhere to these requirements before contacting support legacy task definition (deprecated) for requests using the deprecated instruction format, the task definition name and revision are displayed as read only fields a deprecation notification recommends migrating to the new annotation instructions format these settings cannot be edited within request settings phases tab the phases tab allows you to configure workflow phases for non legacy requests only phases with configurable settings are displayed ⚠️ important all changes to configurations will impact future outcomes exclusively changes will not retroactively affect previously processed inputs general settings workflow checkpoints only available to kognic users, and only for non legacy workflows enable more specific progress tracking for request inputs by defining workflow checkpoints this is an experimental feature production complete checkpoint — defines when an input is considered to have completed the production flow setting description checkpoint placement select which workflow phase the checkpoint should be placed before effect when an input passes through this checkpoint, it becomes available to the client for quality assurance default if not defined, the checkpoint is placed before the delivery phase ⚠️ important once saved, the checkpoint placement cannot be edited a critical warning is displayed before saving, and after saving, the field becomes read only with a locked indicator phase settings each workflow phase has specific configurable options the available settings depend on the phase type common settings these settings appear across multiple phase types setting description available in keep reviewer throughout phase follow up reviews are automatically assigned to the person who originally reviewed the input in this phase sampled review, full review, client qa, delivery assign corrections to last input assignee when a review is rejected, the correction task is assigned to the team member who submitted the last task before the review, regardless of phase sampled review, full review, client qa, delivery allow team member to perform consecutive tasks allows the same person to perform consecutive tasks primarily used for testing purposes all phases focused review for corrections tasks include a filtered list of objects that received feedback for review rounds 2+ tasks include objects that addressed feedback and new objects created in correction sampled review, user quality review, expert verification, client qa sampled review phase a review phase where only a percentage of inputs are selected for review sampling rate — set the probability (0 100%) of an input entering the phase being selected for review changes only affect inputs that have not yet entered the phase additional settings available setting description keep reviewer throughout phase follow up reviews assigned to original reviewer assign corrections to last input assignee corrections go to last task submitter allow team member to perform consecutive tasks same person can do sequential tasks allow workspace owner to manage review stage grants owner permission to manage review actions focused review filters objects in correction and follow up reviews prevent editing of annotations in review tasks annotations can be viewed but not edited in review feedback can still be added and edited this applies to all ongoing review tasks full review phase a review phase where all inputs are reviewed setting description allow team member to perform consecutive tasks same person can do sequential tasks keep reviewer throughout phase follow up reviews assigned to original reviewer assign corrections to last input assignee corrections go to last task submitter quality review phase a quality assurance review phase for monitoring annotator performance setting description focused review filters objects in correction and follow up reviews expert verification phase a verification phase for expert level quality checks setting description focused review filters objects in correction and follow up reviews client qa review phase a quality assurance phase for client review of annotations sampling rate — set the probability (0 100%) of an input being selected for client qa changes only affect inputs that have not yet entered the phase additional settings available setting description keep reviewer throughout phase follow up reviews assigned to original reviewer assign corrections to last input assignee corrections go to last task submitter allow team member to perform consecutive tasks same person can do sequential tasks focused review filters objects in correction and follow up reviews correction phase a phase where annotators address feedback from review stages setting description allow team member to perform consecutive tasks same person can do sequential tasks delivery phase the final phase, where completed annotations are delivered setting description allow team member to perform consecutive tasks same person can do sequential tasks keep reviewer throughout phase follow up reviews assigned to original reviewer assign corrections to last input assignee corrections go to last task submitter setting description available in keep reviewer throughout phase follow up reviews are automatically assigned to the person who originally reviewed the input in this phase sampled review, full review, client qa, delivery assign corrections to last input assignee when a review is rejected, the correction task is assigned to the team member who submitted the last task before the review, regardless of phase sampled review, full review, client qa, delivery allow team member to perform consecutive tasks allows the same person to perform consecutive tasks primarily used for testing purposes all phases focused review for corrections tasks include a filtered list of objects that received feedback for review rounds 2+ tasks include objects that addressed feedback and new objects created in correction sampled review, user quality review, expert verification, client qa