Guide: Run a P2P coaching study

A P2P coaching study pairs one fast annotator with a small group — typically 3 to 6 people — for a couple of weeks, and measures whether the group's Annotate Speed moves. This guide walks the whole round.

Open your project and go to Metrics ▸ Productivity. The user breakdown table at the bottom carries the numbers the choice is made on: Annotate Speed, correction ratio, time allocation, days worked and Training sessions.

In Advanced filters, narrow to the people worth coaching — for example Training sessions "less than 2 sessions" to skip anyone who has been coached recently, and Reported time "more than 8 hours" to skip anyone with too little activity to judge. Sort by Annotate Speed to find the slowest of what is left.

Hover a Training sessions count to see which rounds someone was already in, and whether they are booked into one that hasn't started yet, shown as (+n).

Tick the users you picked. The batch bar reads "N users selected" and offers Create study — click it and the create dialog opens with those people already added as members.

Fill in:

Select Create. You can also start from the Study tab with Create new study and pick members by name there.

Open the study from the Study tab. The cards at the top show each metric and how far it has moved since the study started; Annotate Speed is the one to watch, and it also has a chart per member. The charts plot the group against the project baseline — everyone else in the project — so a project-wide change doesn't read as coaching.

A study needs at least 2 working days of data before it shows a change, and a change only gets a colour when it is bigger than the group's normal day-to-day movement.

Use Manage members to add or remove people while the study runs. The same Training sessions column is shown there, excluding the study you opened it from.

Back on Metrics ▸ Productivity, everyone from the finished study now counts one session higher, so the "less than 2 sessions" shortlist moves on to the next group by itself. On the Study tab, filter the list by Type, Coach or Status to find earlier rounds and compare.

This guide plans one round inside one project. If you are responsible for coaching across an organization, start from Team Management ▸ P2P Sessions instead.

Its Coverage tab lists every active project with the gap between its fastest and slowest fifth of annotators, whether a session is expected there, and whether one is already running — so the choice of project is made from the numbers rather than from memory. Auto-suggest on a row opens a planner that composes the groups and creates a study for each of them in one go, which replaces steps 1 and 2 above when a project needs more than one group.

Reviews due then tracks which studies report this week, and Impact carries what previous rounds changed, per project and per coach.