Action assist: 3D boxes

workflow find an object in the point cloud does it have a white/peach prediction box? select it do you get a go to best frame suggestion? press it then select the desired class did you not get a best frame suggestion? select the desired class no prediction box? draw the object manually with center/side mode check if the object is stationary and set the property enable aggregation with alt+a and scrub \ the object is smeared out and appears to leave the location > not stationary \ the object points are not smeared out, except for possible drift > is stationary after you have created the object you will either annotate a stationary object or a dynamic object and they will have different workflows the dynamic object workflow find the object in the next frame move the object to the next frame if the predictions are properly tracked, accept prediction shall be used otherwise utilize quick move docid\ tqyvley1f46xww8adbuss make necessary adjustments are there any visible ground points nearby? if not, do not update pitch/roll as it will only waste time for no value being added translate the box to include any missed points if you created the object in the best frame with a prediction you should not have to update the size of the cuboid if you need to rescale, you did not annotate the object in the best frame go to the next frame with the go to next frame action ( shift+space ) and search for points near the highlight box or the current box are there any points belonging to the object visible? no go to step 4 if you have repeated step 4 & 5 for 2 5 consecutive frames and points are not found trim lifespan yes go to step 6 move the object to the next location is the object temporarily stopped (e g at a red light) and all points are included? move on (step 4) does the predicted box correctly include all points? accept it shift+click or press r the prediction is completely off quick move the object to the new location make only the necessary adjustments go to step 4 the stationary object workflow aggregate the point cloud (alt+a) use go to best predicted frame and move keyframe according to action assist set the correct rotation of the box while you only have this one keyframe identify drift no drift identified size the box to include all points use filter cuboids below point threshold docid\ mlkqv7jin eozifqosomb you are now done with this object point cloud drift itendified size the box roughly to the correct size with aggregation turned on was this the first object within a 50 meter radius? yes change frame until points are falling far outside of the box make the smallest translate to include the points never update rotation step to the middle point between the last two keyframes, add a keyframe if needed repeat step 1 3 until done no go to the next predicted location with shift+space and add a keyframe with accept prediction ( r ) make the smallest possible translate to include points if any are missed suggested actions go to next frame shift+space jumps to and zooms into the next frame that needs a keyframe, framing the camera around the predicted box marks the previous frame as visited accept prediction r accepts the predicted box in the current frame as a keyframe go to best predicted frame offered when you've just selected an object that has no annotation yet, or when the existing keyframe isn't on the best frame for the object jumps to the frame where the prediction is clearest (best point coverage) so you can start there move keyframe offered for stationary objects right after the first keyframe replaces the first keyframe with one placed on the best predicted frame, so the rest of the track inherits a cleaner anchor the stationary toggle if the task is not configured to include the 'is stationary' property you will get a suggestion to set the object as stationary in the action assist window auto accept toggle auto accept in the action assist panel to accept predictions as you navigate, so a single shift+space accepts the current suggestion and moves to the next frame in one step useful when the predictions look consistently good and you know many keyframes are required toggle location is subject to change the highlight box the green box you see at the current frame is the assistant's current prediction for the selected object it is not the object itself — it's a preview shift+click the highlight box to snap the selected object onto the prediction at that frame the highlight is hidden when predictions are toggled off globally, except outside the keyframe range where it still appears to help you find the object the solid line out from the box is the tracked trajectory frames where predictions are used will be connected by the solid line the dashed line is the extrapolated portion up to five frames before the first and after the last tracked frame treat dashed sections as best effort stationary objects don't show a trajectory things to know you do not have to select predictions as you annotate the object any prediction overlapping the current trajectory will automatically be picked up by the system you only have to follow the highlight box the assist walks forward from your last keyframe until the end of the sequence, then backward from your first keyframe to the start — stopping automatically at frames marked first/last for the object visited frames are remembered once you step past a frame, the assist won't loop back to it for the same object switching objects resets the visited list for the new object a single keyframe is usually enough to get a full track the pipeline extrapolates the velocity from your keyframe(s) and matches to overlapping prelabel predictions to fill the rest of the sequence heading alignment only kicks in when the object is moving fast enough and isn't turning sharply slow moving objects (parking maneuvers, crawling traffic) keep the heading from the prediction or your last keyframe — verify the heading on these manually for objects that should be treated as fully stationary, use the stationary toggle instead predictions already used by other objects are skipped predictions that another annotation has consumed, or that you've explicitly dismissed via the copilot ignore list, will not be offered as matches a suggestion only appears when there are predictions to extrapolate to, or when the trajectory clearly continues into a frame if the object exits the scene, becomes occluded for a long stretch, or has no nearby prelabels, no suggestion is generated for that frame — mark the last visible frame as the object's final frame and the assist will stop walking forward ground alignment uses the per frame ground mesh under the cuboid in areas where the lidar is sparse and the mesh is unreliable, the pitch/roll correction is skipped and the prediction keeps its original orientation for stationary objects, the relative keyframe predictor only uses anchors within 50 m that are themselves marked stationary and have keyframes covering both the reference frame and the prediction frame adding stationary annotations to nearby objects (signs, parked cars) materially improves the assist for everything else stationary in the scene related content 3d boxes docid\ jw2jfj5i04upvcc7hv3ek action assist 2d / 3d fusion docid\ take58ec1rmpcbldkzrn4 copilot docid 1kfjv7 embnrkxrv8offt side mode docid\ zdkgrbvxd6uwmf17c0fwh center mode docid\ e7wp2mdimgjfihcvj1ma7 filter cuboids below point threshold docid\ mlkqv7jin eozifqosomb point cloud aggregation docid\ bcoytptdi72b6fmuqvntx