Aggregated Lidars and Cameras Sequence

2 min

Aggregation is moving to the Task Definition/Annotation Instruction.

Starting with Kognic IO v2.9.0, support for aggregated scenes is changing.

The Annotation Instruction (formerly Task Definition) can now require that annotation be performed using an aggregated view, even if the scene itself is not an Aggregated Lidars and Cameras Sequence.

We have made it possible to use standard Lidars and Cameras Sequence scenes in these cases, removing the need to specifically create Aggregated Lidars and Camera Sequence scenes as documented below.

This approach allows one Lidars and Cameras Sequence scene to be annotated multiple times in a mix of aggregated or sequential setups, optionally using different pre-annotations.

Please contact Kognic for more information about this way of working.

An AggregatedLidarsAndCamerasSeq scene consists of a sequence of camera images and lidar point clouds, where each frame consists on 1-20 camera images as well as 1-20 point clouds (in the case where you have pre-aggregated your point clouds, the first frame consists of 1-20 point clouds and all other frames 0 point clouds). What differentiates AggregatedLidarsAndCamerasSeqfrom LidarsAndCamerasSeq is that point clouds are aggregated over time during annotation which results in one big point cloud in the coordinate system of the first frame. Therefore, ego motion data is mandatory for this type of scene. For more documentation on what each field corresponds to in the AggregatedLidarsAndCamerasSeq object please check the section related to Scene Overview.

Refer to Coordinate Systems for more information about what coordinate systems to use.

from __future__ import absolute_import

from datetime import datetime
from typing import Optional
from uuid import uuid4

import kognic.io.model.scene.aggregated_lidars_and_cameras_seq as ALCSM
from examples.calibration.calibration import create_sensor_calibration
from kognic.io.client import KognicIOClient
from kognic.io.logger import setup_logging
from kognic.io.model import CreateSceneResponse, EgoVehiclePose, Image, PointCloud, Position, RotationQuaternion


def run(client: KognicIOClient, dryrun: bool = True, **kwargs) -> Optional[CreateSceneResponse]:
    print("Creating Lidar and Camera Sequence Scene...")

    lidar_sensor1 = "lidar"
    cam_sensor1 = "RFC01"
    cam_sensor2 = "RFC02"
    metadata = {"location-lat": 27.986065, "location-long": 86.922623, "vehicle_id": "abg"}

    # Create calibration
    # (Please refer to the API documentation about calibration for more details)
    calibration_spec = create_sensor_calibration(
        f"Collection {datetime.now()}",
        [lidar_sensor1],
        [cam_sensor1, cam_sensor2],
    )
    created_calibration = client.calibration.create_calibration(calibration_spec)

    scene = ALCSM.AggregatedLidarsAndCamerasSequence(
        external_id=f"Aggregated-LCS-full-example-{uuid4()}",
        frames=[
            ALCSM.Frame(
                frame_id="1",
                relative_timestamp=0,
                point_clouds=[
                    PointCloud(
                        filename="./examples/resources/point_cloud_RFL01.las",
                        sensor_name=lidar_sensor1,
                    ),
                ],
                images=[
                    Image(
                        filename="./examples/resources/img_RFC01.jpg",
                        sensor_name=cam_sensor1,
                    ),
                    Image(
                        filename="./examples/resources/img_RFC02.jpg",
                        sensor_name=cam_sensor2,
                    ),
                ],
                metadata={"dut_status": "active"},
                ego_vehicle_pose=EgoVehiclePose(
                    position=Position(x=1.0, y=1.0, z=1.0),
                    rotation=RotationQuaternion(w=0.01, x=1.01, y=1.01, z=1.01),
                ),
            ),
            ALCSM.Frame(
                frame_id="2",
                relative_timestamp=500,
                point_clouds=[
                    PointCloud(
                        filename="./examples/resources/point_cloud_RFL02.las",
                        sensor_name=lidar_sensor1,
                    ),
                ],
                images=[
                    Image(
                        filename="./examples/resources/img_RFC11.jpg",
                        sensor_name=cam_sensor1,
                    ),
                    Image(
                        filename="./examples/resources/img_RFC12.jpg",
                        sensor_name=cam_sensor2,
                    ),
                ],
                ego_vehicle_pose=EgoVehiclePose(
                    position=Position(x=2.0, y=2.0, z=2.0),
                    rotation=RotationQuaternion(w=0.01, x=2.01, y=2.01, z=2.01),
                ),
            ),
            ALCSM.Frame(
                frame_id="3",
                relative_timestamp=1000,
                point_clouds=[
                    PointCloud(
                        filename="./examples/resources/point_cloud_RFL02.csv",
                        sensor_name=lidar_sensor1,
                    ),
                ],
                images=[
                    Image(
                        filename="./examples/resources/img_RFC11.jpg",
                        sensor_name=cam_sensor1,
                    ),
                    Image(
                        filename="./examples/resources/img_RFC12.jpg",
                        sensor_name=cam_sensor2,
                    ),
                ],
                ego_vehicle_pose=EgoVehiclePose(
                    position=Position(x=3.0, y=3.0, z=3.0),
                    rotation=RotationQuaternion(w=0.01, x=2.01, y=2.01, z=2.01),
                ),
            ),
        ],
        calibration_id=created_calibration.id,
        metadata=metadata,
    )
    # Create scene
    return client.aggregated_lidars_and_cameras_seq.create(scene, dryrun=dryrun, **kwargs)


if __name__ == "__main__":
    setup_logging(level="INFO")
    client = KognicIOClient()

    # Project - Available via `client.project.get_projects()`
    project = "<project-identifier>"

    run(client, project=project)

Use dryrun to validate scene

Setting dryrun parameter to true in the method call, will validate the scene using the API but not create it.

Reuse calibration

Note that you can, and should, reuse the same calibration for multiple scenes if possible.