Annotation Points
Annotation Points (AP) Overview
What are Annotation Points? Annotation Points (AP) are a project-level scoring system that quantifies every annotation actionβtasks, objects, shapes, and propertiesβinto a single, normalized unit.
For setting AP see our Annotation Points & Targets guide for details.
1. Why use AP?
- Normalize effort. Different projects and taxonomies vary in complexity; AP lets you compare βapples to apples.β
- Set clear targets. Define an AP/hour goal per project and track productivity as a percentage of that target.
- Align on value. Reward high-effort work (3D boxes, property setting etc..) and discourage low value work such as micro adjustments.
2. How is AP calculated?
AP Calculation Formula
- Points Per Task: flat bonus for submitting a task
- Points Per Object: base points for creating each object
- Points Per Shape: additional points per frame/shape instance
- Points Per Property: points for setting or updating object/scene properties
- Points Per Static Objects: Points for setting or updating a static object
- Points Per Dynamic Object: Points for setting or updating a dynamic object.
AP are only generated during annotation and correction tasks (not pure reviews). Actual AP appear in parentheses next to your productivity percentage.
AP Targets & Productivity %
Once your project's AP specification is configured, set an AP target per reported hour (APT, e.g. 300 AP/h). It covers all reported work on the project, not only annotating. Productivity is then:
- π’ β₯ 100%: meeting or exceeding target
- π΄ < 100%: below target
Stopwatch-reported hours are used for βHours worked.β If no stopwatch data exists, Productivity shows as N/A.
Annotators are also measured on annotate speed, AP per hour while annotating, against a separate annotate speed target (by default APT Γ 1.3). See Annotation Points and TargetAnnotation Points and Target. Projects can also configure an expected learning curve β a per-week speed ramp-up expressed as a percentage of the annotate speed target. This is used on the Learning Curve metrics tab to visualize whether new annotators are onboarding at the expected pace.
3. FAQs
Q: Why did my AP drop, or even go negative when I added shapes?
A: Our system takes a βsnapshotβ of your total AP every 10 minutes and recalculates the entire annotation chain using the current point weights. If the Project Manager have recently lowered the points per object/shape, the snapshot will:
- Apply the new lower weights retroactively to all previously annotated shapes.
- Compare that new total against your last snapshot.
If the new weights are lower than before, your historic AP total shrinksβso adding shapes can produce a negative delta in AP.