--- title: Action assist: 2D / 3D fusion slug: task-view-guide/action-assist-2d-3d-fusion docTags: createdAt: 2025-05-16T09:19:34.196Z --- In 2D/3D fusion tasks where you connect 3D boxes and 2D boxes, Action Assist provides recommendations in the 2D view. It is most efficient to annotate an object in the 3D view first. This provides a perfect prior for where the object is in all cameras at once. :::::ExpandableHeading # Sequence First you want to annotate the object fully in 3D using [Action assist: 3D boxes](docId\:SbxA8fFxzbll_5FWq7Qnl). Once finished, switch over to the 2D view. Here you will be guided through the currently selected camera to each frame where the 3D object projects into the camera image, marked with the blue "attention frame" markers in the videobar. ![](https://api.archbee.com/api/optimize/wLEz_RbWP6nIMpY-prjlm/Z4duuKVfFsGy1DBwuXk-f_image.png) :::hint{type="danger"} In order to be productive you must **avoid** the [Create connected 2D box](docId:8OxMj9RnUV9lI-8VH1Bdx) feature, the projection is always too large - making you have to adjust every single 2D box. Action assist learns how to correct for calibration errors based on the annotated keyframes. It will also make action-assist stop giving you suggestions and predictions. ::: ### Workflow ::::WorkflowBlock :::WorkflowBlockItem Annotate the object in 3D as usual ([Action assist: 3D boxes](docId\:SbxA8fFxzbll_5FWq7Qnl)) ::: :::WorkflowBlockItem Switch to the camera view ::: :::WorkflowBlockItem Press `shift+space` to jump to the first frame that needs a box, then `R` to accept the suggestion — or turn on Auto-accept and just press `shift+space` through the sequence. ::: :::WorkflowBlockItem Adjust any boxes that don't look right; subsequent suggestions in nearby frames will follow your adjustment. ::: :::WorkflowBlockItem Repeat step 3 and 4 until the video bar shows no more highlighted frames for the object. Verify that the start and end frames are marked correctly. ::: :::: ### How it works 1. Select an object that already has at least one 3D cuboid annotation. 2. Open a camera view. For every frame where the object's cuboid appears, the system projects a 2D bounding box into that image. 3. If there is an overlapping prediction of similar size, that prediction is used instead of the raw projection — giving you a tighter, more accurate box. ### Suggested actions The Action Assist panel shows the next recommended step for the selected object. 1. Go to next frame - `shift+space` - Jumps to and zooms into the next frame that needs a box. Marks the previous frame as visited. 2. Accept prediction - `R` - Accepts the suggested box in the current frame as a keyframe. ### Auto-accept Toggle Auto-accept in the Action Assist panel to accept predictions as you navigate, so a single `shift+space` accepts the current suggestion and moves to the next frame in one step. Useful when the suggestions look consistently good and you only need to step through to confirm. Toggle location is subject to change. ::Image[]{src="https://api.archbee.com/api/optimize/wLEz_RbWP6nIMpY-prjlm/luzYtthk7Y2cbZUdi_EhS_image.png" size="40" width="340" height="227" position="center" darkWidth="340" darkHeight="227" showCaption="false"} ### Things to know The assist walks forward from your last keyframe until the end of the sequence, then backward from your first keyframe to the start — stopping automatically at frames marked first/last for the object. - Visited frames are remembered. Once you step past a frame, the assist won't loop back to it for the same object. Switching objects resets the visited list for the new object. - Larger objects (cars, trucks, buses, etc.) use a tighter mesh-based projection rather than the raw cuboid corners, so the suggested box hugs the vehicle more closely. - If you nudge a box, nearby suggestions follow. When you edit a 2D box away from where the cuboid projects to, the assist remembers the angular offset and applies it to suggestions in nearby frames (within \~3 seconds) — useful when the camera/lidar calibration drifts persistently in one direction. The compensation only kicks in away from the image edges where the geometry is reliable. - Predictions already used by other objects are skipped. Predictions that another annotation has consumed, or that you've explicitly dismissed via the Copilot ignore list, will not be offered as matches. - A suggestion only appears when the object has a 3D cuboid in that frame. If the cuboid is missing in a particular frame, no 2D suggestion is generated for that frame. ::::: :::ExpandableHeading # Single frame The single frame action assist for 2D / 3D fusion will guide you through the optimal workflow. We will be annotating the 3D box first, then draw the object in 2D in each source that the 3D box is visible in. 1. **Next prediction in 3D** You will be guided to the next prelabel in the 3D scene. If it's a false positive you can simple press 'backspace' to ignore it and press `shift+space` to go to the next one. 2. **Select class** Selecting a class with a prelabel selected will create an object at it's location. If there is no prelabel for a given object - draw it manually. 3. **Go to 2D** While you have the context for the current object we encourage you to finish annotating 2D before drawing the next object in 3D. 4. **Draw connected 2d box** You will be zoomed to the location of the projection of the 3D box, and the drawing tool for [2D bounding boxes](docId\:TfjsznROmWyAZm4_SMR5C) will activate. If the object is fully occluded press `shift+space` again to skip this source. This step will be repeated for each camera source the object is visible in. 5. **Go back to 3D** This action will deselect the current object and go back to 3D. Going back to step 1. Now putting it all together we get the workflow illustrated below. ![](https://api.archbee.com/api/optimize/wLEz_RbWP6nIMpY-prjlm/8Cc_VSBvUuDORgRuiWwdo_action-assist-sf.gif) ::: *** ## Related content ::::LinkArray{contentSource="CUSTOM"} :::LinkArrayItem [3D boxes](docId\:jw2JFJ5i04upVcc7hv3Ek) ::: :::LinkArrayItem [2D bounding boxes](docId\:TfjsznROmWyAZm4_SMR5C) ::: :::LinkArrayItem [Action assist: 3D boxes](docId\:SbxA8fFxzbll_5FWq7Qnl) ::: :::LinkArrayItem [Create connected 2D box](docId:8OxMj9RnUV9lI-8VH1Bdx) ::: ::::