Cuboid Copilot

The predictions are marked with dashed white outlines in the image. They can be selected by clicking on them. To deselect, press escape

If the required data is available - the trajectory of the prediction over it's lifetime will also be visualized to guide with how the object is moving in the scene.

There are two ways of 'drawing' a new object using the predictions and the cuboid copilot. The first way is that you select the class you wish to create, which activates the cuboid copilot drawing tool afterwards you can click on a prediction to create an object at it's location.

The second method in which you can create a new object from a prediction is that you select the prediction first by clicking on it. As soon as you select a class, an object will be created at the location without any further action. See gif below. Select prediction, select class

While editing and you wish to update the object's location with that of the prediction you have two options. This will update the heading and position. The size as well as pitch and roll will not change.

Shift + click on the prediction

Select a prediciton and press the 'copilot accept' shortcut (R by default).

You are also able to interact with the prediction when working in grid-view, if you prefer that over the main-view.

When you have a prediction selected at the same time as editing an object you will be able to use the 'copilot step' feature. The shortcut is Enter. This will assist you in guiding you to where your attention is needed the most. It will guide you a bit differently depending if you are in a single frame task or a sequence task.

Using the 'copilot step' function in a single frame task will select the next prediction that is the closest to the current selected object. If it's an actual object then you can simply select a class and an object will be created, as the drawing process describes.

However if it's a false positive detection or something that should not be annotated, you can simply press the 'copilot step' shortcut to skip the prediction. A skipped prediction is indicated by a dark grey color. You are still able to interact with skipped predictions if it was done by mistake.

Using the 'copilot step' function in sequences you will be guided to the 'worst frame' for the selected object. If needed use the copilot editing process, simply press 'copilot accept' shortcut (default R) is recommended and edit the object further only if required. Then press the shortcut again and continue. When you have checked all frames you will be notified and can continue with the next object ✅.

When a prediction is selected, the ‘show best frame’ feature will be available to you in sequences. It will find the frame that contains the most information for you to estimate the objects size. This frame will be indicated with a ☆ in the videobar. The copilot will navigate to this frame for you and zoom in on the prediction.

Sometimes you will encounter predictions that should not be annotated or are false positives from the object detection system. These prediction can be erased so that they are less distracting, they will also not be included in the 'copilot stepping'.

To use the eraser, activate the copilot eraser tool and click and hold the left mouse button and mouse over the predictions that should be erased.