라이다 및 카메라 시퀀스
6 분
kognic-io 2.5.0부터는 씬 타입(LidarsAndCamerasSequence)을 노출하지 않고 단일 모델로 모든 씬 생성을 단순화한 새로운 씬 모델 사용을 권장하고 있습니다.
LidarsAndCamerasSequence는 카메라 이미지와 라이다 포인트 클라우드의 시퀀스로 구성되며, 각 프레임은 1-20개의 카메라 이미지와 1-20개의 포인트 클라우드로 이루어집니다. LidarsAndCamerasSequence 객체의 각 필드가 무엇에 해당하는지에 대한 자세한 내용은 개요씬 개요 섹션을 참고하세요.
from __future__ import absolute_import
from datetime import datetime
from typing import Optional
from uuid import uuid4
import kognic.io.model.scene.lidars_and_cameras_sequence as LCSM
from examples.calibration.calibration import create_sensor_calibration
from kognic.io.client import KognicIOClient
from kognic.io.logger import setup_logging
from kognic.io.model import CreateSceneResponse, Image, PointCloud
def run(client: KognicIOClient, dryrun: bool = True, **kwargs) -> Optional[CreateSceneResponse]:
print("Creating Lidar and Camera Sequence Scene...")
lidar_sensor1 = "lidar"
cam_sensor1 = "RFC01"
cam_sensor2 = "RFC02"
metadata = {"location-lat": 27.986065, "location-long": 86.922623, "vehicle_id": "abg"}
# Create calibration
# (Please refer to the API documentation about calibration for more details)
calibration_spec = create_sensor_calibration(
f"Collection {datetime.now()}",
[lidar_sensor1],
[cam_sensor1, cam_sensor2],
)
created_calibration = client.calibration.create_calibration(calibration_spec)
scene = LCSM.LidarsAndCamerasSequence(
external_id=f"LCS-example-{uuid4()}",
frames=[
LCSM.Frame(
frame_id="1",
relative_timestamp=0,
point_clouds=[
PointCloud(
filename="./examples/resources/point_cloud_RFL01.las",
sensor_name=lidar_sensor1,
),
],
images=[
Image(
filename="./examples/resources/img_RFC01.jpg",
sensor_name=cam_sensor1,
),
Image(
filename="./examples/resources/img_RFC02.jpg",
sensor_name=cam_sensor2,
),
],
metadata={"dut_status": "active"},
),
LCSM.Frame(
frame_id="2",
relative_timestamp=100,
point_clouds=[
PointCloud(
filename="./examples/resources/point_cloud_RFL02.las",
sensor_name=lidar_sensor1,
),
],
images=[
Image(
filename="./examples/resources/img_RFC11.jpg",
sensor_name=cam_sensor1,
),
Image(
filename="./examples/resources/img_RFC12.jpg",
sensor_name=cam_sensor2,
),
],
metadata={"dut_status": "active"},
),
],
calibration_id=created_calibration.id,
metadata=metadata,
)
# Create scene
return client.lidars_and_cameras_sequence.create(scene, dryrun=dryrun, **kwargs)
if __name__ == "__main__":
setup_logging(level="INFO")
client = KognicIOClient()
# Project - Available via `client.project.get_projects()`
project = "<project-identifier>"
run(client, project)dryrun으로 씬 검증하기
메서드 호출 시 dryrun 파라미터를 true로 설정하면, API를 사용해 씬을 검증만 하고 실제로 생성하지는 않습니다.
캘리브레이션 재사용하기
가능하다면 여러 씬에 동일한 캘리브레이션을 재사용할 수 있으며, 그렇게 하는 것이 좋습니다.
자차(Ego Vehicle) 모션 정보 제공하기
자차 모션(즉, 자차의 위치 및 회전) 정보는 LidarsAndCamerasSeq를 생성할 때 제공할 수 있는 선택적 정보입니다. 이 정보를 제공하면 정적 객체를 어노테이션하는 데 걸리는 시간을 크게 단축할 수 있습니다. 자차 모션 정보는 씬 내의 각 Frame에 EgoVehicleMotion 객체를 전달하여 제공합니다.
from __future__ import absolute_import
from datetime import datetime
from typing import Optional
from uuid import uuid4
import kognic.io.model.scene.lidars_and_cameras_sequence as LCSM
from examples.calibration.calibration import create_sensor_calibration
from kognic.io.client import KognicIOClient
from kognic.io.logger import setup_logging
from kognic.io.model import CreateSceneResponse, EgoVehiclePose, Image, PointCloud, Position, RotationQuaternion
def run(client: KognicIOClient, dryrun: bool = True, **kwargs) -> Optional[CreateSceneResponse]:
print("Creating Lidar and Camera Sequence Scene...")
lidar_sensor1 = "lidar"
cam_sensor1 = "RFC01"
cam_sensor2 = "RFC02"
cam_sensor3 = "RFC03"
metadata = {"location-lat": 27.986065, "location-long": 86.922623, "vehicle_id": "abg"}
# Create calibration
calibration_spec = create_sensor_calibration(f"Collection {datetime.now()}", [lidar_sensor1], [cam_sensor1, cam_sensor2, cam_sensor3])
created_calibration = client.calibration.create_calibration(calibration_spec)
scene = LCSM.LidarsAndCamerasSequence(
external_id=f"LCS-full-example-{uuid4()}",
frames=[
LCSM.Frame(
frame_id="1",
relative_timestamp=0,
point_clouds=[
PointCloud(filename="./examples/resources/point_cloud_RFL01.las", sensor_name=lidar_sensor1),
],
images=[
Image(filename="./examples/resources/img_RFC01.jpg", sensor_name=cam_sensor1),
Image(filename="./examples/resources/img_RFC02.jpg", sensor_name=cam_sensor2),
],
metadata={"dut_status": "active"},
ego_vehicle_pose=EgoVehiclePose(
position=Position(x=1.0, y=1.0, z=1.0), rotation=RotationQuaternion(w=0.01, x=1.01, y=1.01, z=1.01)
),
),
LCSM.Frame(
frame_id="2",
relative_timestamp=500,
point_clouds=[
PointCloud(filename="./examples/resources/point_cloud_RFL02.las", sensor_name=lidar_sensor1),
],
images=[
Image(filename="./examples/resources/img_RFC11.jpg", sensor_name=cam_sensor1),
Image(filename="./examples/resources/img_RFC12.jpg", sensor_name=cam_sensor2),
],
ego_vehicle_pose=EgoVehiclePose(
position=Position(x=2.0, y=2.0, z=2.0), rotation=RotationQuaternion(w=0.01, x=2.01, y=2.01, z=2.01)
),
),
],
calibration_id=created_calibration.id,
metadata=metadata,
)
# Create scene
return client.lidars_and_cameras_sequence.create(scene, dryrun=dryrun, **kwargs)
if __name__ == "__main__":
setup_logging(level="INFO")
client = KognicIOClient()
# Project - Available via `client.project.get_projects()`
project = "<project-identifier>"
run(client, project=project)좌표계
자차 포즈의 position과 rotation은 모두 로컬 좌표계를 기준으로 한다는 점에 유의하세요.
셔터 타이밍
Frame 내의 Image에는 셔터 타이밍을 제공할 수 있습니다.
셔터 타이밍은 나노초 단위의 유닉스 타임스탬프 값 쌍이며, 셔터 시작 시각과 종료 시각 두 값 모두 반드시 제공되어야 합니다. 각 프레임의 각 이미지마다 지정됩니다.
from __future__ import absolute_import
import os.path
from datetime import datetime
from typing import Optional
from uuid import uuid4
import kognic.io.model.scene.lidars_and_cameras_sequence as LCSM
from examples.calibration.calibration import create_sensor_calibration
from examples.imu_data.create_imu_data import create_dummy_imu_data
from kognic.io.client import KognicIOClient
from kognic.io.logger import setup_logging
from kognic.io.model import CreateSceneResponse, Image, ImageMetadata, PointCloud
def run(client: KognicIOClient, dryrun: bool = True, **kwargs) -> Optional[CreateSceneResponse]:
print("Creating Lidar and Camera Sequence Scene...")
lidar_sensor1 = "RFL01"
lidar_sensor2 = "RFL02"
cam_sensor1 = "RFC01"
cam_sensor2 = "RFC02"
metadata = {"location-lat": 27.986065, "location-long": 86.922623, "vehicleId": "abg"}
examples_path = os.path.dirname(__file__)
# Create calibration
calibration_spec = create_sensor_calibration(f"Collection {datetime.now()}", [lidar_sensor1, lidar_sensor2], [cam_sensor1, cam_sensor2])
created_calibration = client.calibration.create_calibration(calibration_spec)
# Generate IMU data
ONE_MILLISECOND = 1000000 # one millisecond, expressed in nanos
start_ts = 1648200140000000000
end_ts = start_ts + 10 * ONE_MILLISECOND
imu_data = create_dummy_imu_data(start_timestamp=start_ts, end_timestamp=end_ts, samples_per_sec=1000)
scene = LCSM.LidarsAndCamerasSequence(
external_id=f"LCS-full-with-imu-and-shutter-example-{uuid4()}",
frames=[
LCSM.Frame(
frame_id="1",
unix_timestamp=start_ts + ONE_MILLISECOND,
relative_timestamp=0,
point_clouds=[
PointCloud(filename=examples_path + "/resources/point_cloud_RFL01.csv", sensor_name=lidar_sensor1),
PointCloud(filename=examples_path + "/resources/point_cloud_RFL02.csv", sensor_name=lidar_sensor2),
],
images=[
Image(
filename=examples_path + "/resources/img_RFC01.jpg",
sensor_name=cam_sensor1,
metadata=ImageMetadata(
shutter_time_start_ns=start_ts + 0.5 * ONE_MILLISECOND, shutter_time_end_ns=start_ts + 1.5 * ONE_MILLISECOND
),
),
Image(
filename=examples_path + "/resources/img_RFC02.jpg",
sensor_name=cam_sensor2,
metadata=ImageMetadata(
shutter_time_start_ns=start_ts + 0.5 * ONE_MILLISECOND, shutter_time_end_ns=start_ts + 1.5 * ONE_MILLISECOND
),
),
],
),
LCSM.Frame(
frame_id="2",
unix_timestamp=start_ts + 5 * ONE_MILLISECOND,
relative_timestamp=4,
point_clouds=[
PointCloud(filename=examples_path + "/resources/point_cloud_RFL11.csv", sensor_name=lidar_sensor1),
PointCloud(filename=examples_path + "/resources/point_cloud_RFL12.csv", sensor_name=lidar_sensor2),
],
images=[
Image(
filename=examples_path + "/resources/img_RFC11.jpg",
sensor_name=cam_sensor1,
metadata=ImageMetadata(
shutter_time_start_ns=start_ts + 4.5 * ONE_MILLISECOND, shutter_time_end_ns=start_ts + 5.5 * ONE_MILLISECOND
),
),
Image(
filename=examples_path + "/resources/img_RFC12.jpg",
sensor_name=cam_sensor2,
metadata=ImageMetadata(
shutter_time_start_ns=start_ts + 4.5 * ONE_MILLISECOND, shutter_time_end_ns=start_ts + 5.5 * ONE_MILLISECOND
),
),
],
),
],
calibration_id=created_calibration.id,
metadata=metadata,
imu_data=imu_data,
)
# Create scene
return client.lidars_and_cameras_sequence.create(scene, dryrun=dryrun, **kwargs)
if __name__ == "__main__":
setup_logging(level="INFO")
client = KognicIOClient()
# Project - Available via `client.project.get_projects()`
project = "<project-id>"
run(client, project=project)역방향 롤링 셔터
Kognic IO 2.17.0부터 사용 가능합니다.
셔터 타이밍이 설정된 경우, 카메라의 롤링 셔터가 아래에서 위로 진행되는 경우를 위해 Images에 True로 설정할 수 있는 reversed_shutter 플래그가 있습니다. 예를 들어 카메라가 뒤집힌 상태로 장착된 경우, 회전된 이미지의 상단 행의 픽셀이 하단 행의 픽셀보다 나중에 촬영된 것입니다. 어노테이션 도구에서 최상의 투영 품질을 얻으려면 도구가 이 차이를 알고 있어야 합니다.
ImageMetadata(
shutter_time_start_ns=1775128684200000000,
shutter_time_end_ns=1775128685000000000,
reversed_shutter=True
)