Team Overview

This view is available for all people given the role of Team Manager. It shows the productivity of your organization's annotators. The view automatically updates every 10 minutes.

Select a time interval filter or sort it on a project or request level.

This view (as well as some other views) use the term Annotation Points, which is a metric used to describe the value of annotation data.

This is configured on a project level, a project determine the value of the shapes, properties and objects that uploaded data will be annotated with. It also discuss expectations or annotation point targets, which is a number that details how many annotation points a user is expected to produce per hour.

This section summarize a few key metrics that are visible in the current selection:

This section describes the different columns present in the table.

This is the name and email of the given user. You are able to click on the name of a user to be taken to another view to gain even more insights into their performance.

The number of currently ongoing tasks that the users are assigned to.

This column details how productive the the listed users are in the selected timespan in comparison with the expected productivity. This metric is displayed as a percentage which is colored green if the user is exceeding expectations, and red if the user is not. Inside the parenthesis the actual number of produces annotation points is displayed. As an example, if a user has worked 2h in a selected time period, they have produced 100APs in this period, and the target is 100AP/h, the user will have a productivity of 50%. The productivity metric is computed using the stopwatch time reporting feature, is this feature is not used this view will not contain productivity metrics, instead a N/A will be displayed.

This is how many tasks that the user has completed during the selected timespan.

In order to be able to understand if a user is producing data with an acceptable quality (not just at an expected rate), this column estimates the quality of the users' work. The quality estimate is presented as a percentage and it's computed by subtracting the ratio of the number of reported feedback items and the number of reviewed objects from 100. For example, if a user has produced 100 objects in 2 tasks, a reviewer has examined both of these tasks and reported 20 issues using the feedback tooling, the quality estimate will be 80%. If no tasks that the user has completed in the selected timespan, no quality metric will be available. This is a crude metric that should not be used for other purposes than ensuring that users are not working at an unsustainable pace. Please note that this column will not contain any valuable data if the users' work is not being reviewed in a way that utilize the feedback tooling to report issues.

This is the number of tasks that the user has completed within the selected timespan that has been reviewed. The higher this number is, the more trustworthy the Quality estimate from the previous section is.

How much time has been reported from Stopwatch during the selected timespan. If no time has been reported, we will fall back to showing the time spent annotating. In this case, a red asterisk will indicate that there is no stopwatch time available, and this will also make it impossible to compute a productivity metric.

The ratio of time allocated to annotation, and the total time reported during the selected timespan. This metric is in this table in order to verify that users are not working too much or too little. If a user does not take any breaks during a workday, this probably indicates a problem.

The time allocation is colored according to the following in order to indicate is a value is acceptable, too high or too low:

0.70 or lower - RED 0.70 - 0.75 - YELLOW 0.75 - 0.85 - WHITE 0.85 - 0.90 - YELLOW 0.90 or higher - RED

This is how many objects that the user has created during the selected timespan. Note, that this might be a negative number, if:

This is how many shapes has created during the selected timespan. Note, that this might be a negative number, if:

For a distinction between objects and shapes, see Objects and Shapes.