--- title: Action assist: 3D boxes slug: task-view-guide/action-assist-3d-boxes docTags: createdAt: 2025-05-16T09:18:49.185Z --- ## Workflow ::::WorkflowBlock :::WorkflowBlockItem ** Find an object in the point cloud** 1. Does it have a white/peach prediction box? Select it. - Do you get a go to best frame suggestion? Press it. Then select the desired class. - Did you not get a best frame suggestion? Select the desired class. 2. No prediction box? Draw the object manually with center/side mode. ::: :::WorkflowBlockItem **Check if the object is stationary** and set the property. Enable aggregation with alt+a and scrub. \- The object is smeared out and appears to leave the location -> Not stationary. \- The object points are not smeared out, except for possible drift -> Is stationary. ::: :::: After you have created the object you will either annotate a stationary object or a dynamic object. And they will have different workflows. :::::ExpandableHeading ### The dynamic object workflow ::::WorkflowBlock :::WorkflowBlockItem Find the object in the next frame. ::: :::WorkflowBlockItem Move the object to the next frame If the predictions are properly tracked, **accept prediction **shall be used. Otherwise utilize [Quick-move](docId\:tQYVlEy1f46XwW8aDBUss). ::: :::WorkflowBlockItem Make necessary adjustments. - Are there any visible ground points nearby? If not, do not update pitch/roll as it will only waste time for no value being added. - Translate the box to include any missed points. - If you created the object in the best frame with a prediction you should not have to update the size of the cuboid. If you need to rescale, you did not annotate the object in the best frame. ::: :::WorkflowBlockItem Go to the next frame with the **Go to next frame** action (`shift+space`) - and search for points near the highlight box or the current box. ::: :::WorkflowBlockItem Are there any points belonging to the object visible? - No. Go to step 4. If you have repeated step 4 & 5 for 2-5 consecutive frames and points are not found. Trim lifespan. - Yes. Go to step 6. ::: :::WorkflowBlockItem Move the object to the next location. 1. Is the object temporarily stopped (e.g. at a red light) and all points are included? Move on (Step 4). 2. Does the predicted box correctly include all points? Accept it. `shift+click` or Press `R`. 3. The prediction is completely off - quick move the object to the new location. ::: :::WorkflowBlockItem Make *only* the necessary adjustments. ::: :::WorkflowBlockItem Go to step 4. ::: :::: ::::: :::::ExpandableHeading ### The stationary object workflow ::::WorkflowBlock :::WorkflowBlockItem Aggregate the point cloud (alt+a) ::: :::WorkflowBlockItem Use **Go to best predicted frame **and **Move keyframe** according to action assist. ::: :::WorkflowBlockItem Set the correct rotation of the box while you only have this one keyframe ::: :::WorkflowBlockItem Identify drift ::: :::WorkflowBlockItem **No drift** **identified** 1. Size the box to include all points. 2. Use [Filter cuboids below point threshold](docId\:mlkQv7jin_EOzifQoSOMB). 3. You are now done with this object. ::: :::WorkflowBlockItem Point cloud **drift itendified** 1. Size the box roughly to the correct size with aggregation turned on. Was this the first object within a 50 meter radius? **Yes** 1. Change frame until points are falling far outside of the box. 2. Make the smallest translate to include the points. Never update rotation. 3. Step to the middle point between the last two keyframes, add a keyframe if needed. Repeat step 1-3 until done. **No** 1. Go to the next predicted location with `shift+space` and add a keyframe with **accept prediction **(`R`) 2. Make the smallest possible translate to include points if any are missed. ::: :::: ::::: ### Suggested Actions - **Go to next frame - **`shift+space` Jumps to and zooms into the next frame that needs a keyframe, framing the camera around the predicted box. Marks the previous frame as visited. - **Accept prediction - **`R` Accepts the predicted box in the current frame as a keyframe. - **Go to best predicted frame** Offered when you've just selected an object that has no annotation yet, or when the existing keyframe isn't on the best frame for the object. Jumps to the frame where the prediction is clearest (best point coverage) so you can start there. - **Move keyframe** Offered for stationary objects right after the first keyframe. Replaces the first keyframe with one placed on the best predicted frame, so the rest of the track inherits a cleaner anchor. - **The stationary toggle** If the task is not configured to include the 'is stationary' property you will get a suggestion to set the object as stationary in the action assist window. ### Auto-accept Toggle Auto-accept in the Action Assist panel to accept predictions as you navigate, so a single `shift+space` accepts the current suggestion and moves to the next frame in one step. Useful when the predictions look consistently good and you know many keyframes are required. Toggle location is subject to change. :::ExpandableHeading ### The highlight box The green box you see at the current frame is the assistant's current prediction for the selected object. It is not the object itself — it's a preview. - `shift+click` the highlight box to snap the selected object onto the prediction at that frame. - The highlight is hidden when predictions are toggled off globally, except outside the keyframe range where it still appears to help you find the object. - The solid line out from the box is the tracked trajectory - frames where predictions are used will be connected by the solid line. - The dashed line is the extrapolated portion - up to five frames before the first and after the last tracked frame. Treat dashed sections as best-effort. - Stationary objects don't show a trajectory. ::: :::ExpandableHeading ### Things to know - You do not have to select predictions. As you annotate the object any prediction overlapping the current trajectory will automatically be picked up by the system. You only have to follow the highlight box. - The assist walks forward from your last keyframe until the end of the sequence, then backward from your first keyframe to the start — stopping automatically at frames marked first/last for the object. - Visited frames are remembered. Once you step past a frame, the assist won't loop back to it for the same object. Switching objects resets the visited list for the new object. - A single keyframe is usually enough to get a full track. The pipeline extrapolates the velocity from your keyframe(s) and matches to overlapping prelabel predictions to fill the rest of the sequence. - Heading alignment only kicks in when the object is moving fast enough and isn't turning sharply. Slow-moving objects (parking maneuvers, crawling traffic) keep the heading from the prediction or your last keyframe — verify the heading on these manually. For objects that should be treated as fully stationary, use the **Stationary** toggle instead. - Predictions already used by other objects are skipped. Predictions that another annotation has consumed, or that you've explicitly dismissed via the Copilot ignore list, will not be offered as matches. - A suggestion only appears when there are predictions to extrapolate to, or when the trajectory clearly continues into a frame. If the object exits the scene, becomes occluded for a long stretch, or has no nearby prelabels, no suggestion is generated for that frame — mark the last visible frame as the object's final frame and the assist will stop walking forward. - Ground alignment uses the per-frame ground mesh under the cuboid. In areas where the lidar is sparse and the mesh is unreliable, the pitch/roll correction is skipped and the prediction keeps its original orientation. - For stationary objects, the relative-keyframe predictor only uses anchors within 50 m that are themselves marked stationary and have keyframes covering both the reference frame and the prediction frame. Adding stationary annotations to nearby objects (signs, parked cars) materially improves the assist for everything else stationary in the scene. ::: *** ## Related content ::::LinkArray{contentSource="CUSTOM"} :::LinkArrayItem [3D boxes](docId\:jw2JFJ5i04upVcc7hv3Ek) ::: :::LinkArrayItem [Action assist: 2D / 3D fusion](docId\:Take58EC1RmPcBLdkZrn4) ::: :::LinkArrayItem [Copilot](docId:1KFJv7-eMBNRKxRV8oFfT) ::: :::LinkArrayItem [Side mode](docId\:ZdkgrBvXD6uwmF17C0FwH) ::: :::LinkArrayItem [Center mode](docId\:e7wp2MdiMgJfIhcVj1ma7) ::: :::LinkArrayItem [Filter cuboids below point threshold](docId\:mlkQv7jin_EOzifQoSOMB) ::: :::LinkArrayItem [Point cloud aggregation](docId\:bcoYTPTDI72b6fMuqvntX) ::: ::::