To access Alignment Analytics, you need one of the following:

You can find the page by navigating to a request in Request Management and clicking View alignment analytics on the request summary tab.

Alignment Analytics shows you how well annotation work aligns with review standards. It gives you a quality picture for a specific request — covering reviewed work, partially reviewed work, and estimates for unreviewed work.

Use this page to:

Each time work passes through a review phase, the system compares the version before and after that phase. The reviewer may accept the work as-is or make corrections — either way, the system records every difference: shapes that were added, removed, or modified, and properties that were changed. These differences are turned into measurable quality metrics.

Think of it as a before-and-after comparison:

The smaller the difference, the higher the quality.

Quality is broken down into four sub-metrics:

Each metric is displayed as a percentage. A score near 100% means almost no corrections were needed.

The page shows three quality cards, each measuring quality at a different stage of the review workflow:

Initial quality

"How much did the first review round need to correct?"

This compares the annotator's original work to the output after the first review. A low score means the reviewer had to make significant corrections. A high score means the annotation was already close to the desired result.

"How good is the work after the first review round?"

This compares the first review round output to the output after a second review. If this score is near 100%, the first review round produced work that needed almost no further corrections.

"What is the estimated quality of everything delivered in this request?"

This combines all available data into a single quality picture. It is the best estimate of the actual quality your customer receives. See Understanding the overall quality estimate for how this is calculated.

Not every assignment gets reviewed. The overall quality metric provides a complete picture by combining three types of data:

The system estimates quality for unreviewed assignments by looking at reviewed work from the same annotator:

If some assignments cannot be estimated, a warning is displayed on the overall quality card.

Every metric comes with a confidence interval that tells you how reliable the number is. The interval gets tighter as more data becomes available.

What to watch for:

Below the quality cards, the page shows the distribution of unreviewed work:

The table lists every unreviewed assignment with:

You can filter the table by Estimation type to focus on specific groups. Right-click any row and select Open in task view... to see the assignment in the annotation tool.

The same estimated quality scores shown in the unreviewed inputs table above are also available directly in the phase inputs table of Quality Review and Expert Verification phases. In those tables, inputs are sorted by estimated quality ascending by default, making it easy to prioritize reviewing the lowest-quality inputs first.

These scores use the lower bound of the confidence interval rather than the point estimate. This conservative approach means the platform shows the worst-case estimate within the confidence range, so a displayed score of 85% means "we are confident the true quality is at least 85%." As more review data accumulates and confidence intervals narrow, the lower bound converges toward the point estimate

Some annotators have no reviewed work anywhere in the project. The system cannot estimate their quality, so those inputs are excluded from the overall score. The warning tells you how many inputs are affected.

This means there is no review data available to estimate quality for that annotator — neither in this request nor elsewhere in the project.

"Unreviewed in request" means the annotator has been reviewed in other requests within the same project — we can still estimate their quality using project sampling. "Unreviewed in project" means the annotator has never been reviewed in this project at all — we cannot estimate their quality.

Review more assignments. As the number of reviewed samples increases, confidence intervals narrow and estimates become more stable. Prioritize reviewing annotators who appear under "Unreviewed in project" since those are your biggest blind spots.