--- title: OpenLABEL Format slug: api-guide/openlabel-format docTags: createdAt: 2024-12-17T12:57:39.872Z --- OpenLABEL is a standardized annotation format developed by [ASAM](https://www.asam.net/). It comes with in-depth [documentation](https://www.asam.net/index.php?eID=dumpFile\&t=f\&f=4566\&token=9d976f840af04adee33b9f85aa3c22f2de4968dd) of the format as well as a [json schema](https://openlabel.asam.net/V1-0-0/schema/openlabel_json_schema.json), which can be used to ensure that the format is valid. Even though OpenLABEL is a strict format there is still some room for interpretation. In this section we try to clarify some of these parts and explain the choices that we have made within the standard. ## Rotation of Cuboids The rotation is such that the y-axis is facing forwards, with a rotation order of `XYZ`. This means that a cuboid with a heading (yaw) equal to 0 is aligned with the y-axis in the positive direction along the axis. This is somewhat different compared to the [ISO 8855 ](https://www.sis.se/api/document/preview/914200/#:~\:text=This%20International%20Standard%20defines%20the,to%20multi%2Dunit%20vehicle%20combinations.)standard, where the forward direction is along the x-axis. Conversion to ISO 8855 can then be done by applying a rotation around the z-axis and changing `sx` and `sy` in the following way ```python import math from typing import List from scipy.spatial.transform import Rotation def convert_to_iso8855(val: List[float]) -> List[float]: """ Converts cuboid values to ISO 8855 """ [x, y, z, qx, qy, qz, qw, sx, sy, sz] = val rotation_1 = Rotation.from_quat([qx, qy, qz, qw]) rotation_2 = Rotation.from_rotvec([0, 0, math.pi / 2]) rot_object = rotation_1 * rotation_2 [qx, qy, qz, qw] = rot_object.as_quat() return [x, y, z, qx, qy, qz, qw, sy, sx, sz] ``` ## Non-sequences are sequences with one frame Due to reasons of simplicity we have made the choice to treat non-sequences in the same way as sequences. This means that non-sequences are represented as a sequence with only one frame. Only data such as `name` and `type` are defined in the top level element keys. All other information is stored under frames, see example below ```json { "objects": { "0": { "name": "car-0", "type": "Car" } }, "frames": { "0": { "objects": { "0": {"object_data": {...}} } } } } ``` ## Stream is just another text property The `stream` property is used to indicate which stream/sensor/source that the geometry and property was annotated in. For example here is an object with a point that has been annotated in a stream with the name `Camera`. Note that all corresponding attributes for the geometry have also been annotated in the same stream. ```json { "object_data": { "point2d": [ { "name": "point-4d2d325f", "val": [300.5300, 286.4396], "attributes": { "text": [ {"name": "stream", "val": "Camera"}, {"name": "Color", "val": "Black"} ] } } ] } } ``` ## Relations We consider two types of relations; unidirectional relations between two objects and group relations. In addition to these, there is a need to represent false relations, i.e. relation properties that are not actually pointers to other objects but rather take values such as `Inconclusive`, `Nothing` or `Unclear`. ### Relations are unidirectional Relations are unidirectional, meaning that if an object, `object1`, has a relation to another object, `object2`, it does not mean that `object2` has a relation to `object1`. Below follows an example where `car-0` is following `car-1` and it is unclear whether `car-2` is following another car or not. ```json { "objects": { "0": {"name": "car-0", "type": "Car"}, "1": {"name": "car-1", "type": "Car"}, "2": {"name": "car-2", "type": "Car"} }, "relations": { "0": { "name": "0", "type": "isFollowing", "rdf_subjects": [{"type": "object", "uid": "0"}], "rdf_objects": [{"type": "object", "uid": "1"}] } } } ``` ### Actions are used to represent false relations In the Kognic Platform, there is support for assigning values to relations that are not actually references to other objects. Examples are `Inconclusive` and `Nothing`. Actions are used to represent these in the following way, where the name of the action determines the value and the type determines the property name. ```json { "objects": { "0": {"name": "lane-0", "type": "Lane"} }, "relations": { "0": { "name": "0", "type": "isSubjectOfAction", "rdf_subjects": [{"type": "object", "uid": "0"}], "rdf_objects": [{"type": "action", "uid": "0"}] } }, "actions": { "0": {"name": "Nothing", "type": "is_pulling_or_pushing"} } } ``` ### Stream specific relations If a relation is stream specific, there will be a property `stream_relations` denoting which stream the list of relations belong to. ```json { // frames.0 // ... "frame_properties": { "streams": { "CAMERA_FRONT": { "description": null, "stream_properties": { "stream_relations": { "1": {} } } } } }, "relations": { "0": {} } } ``` ## Representing polygons Polygons are described by a list of `Poly2d` objects in OpenLABEL. One of these represents the exterior while the others represent potential holes and this is determined by the boolean property `is_hole`. Below follows an example of a polygon with one hole. ```json { "object_data": { "poly2d": [ { "name": "poly1", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "boolean": [{"name": "is_hole", "val": false}] } }, { "name": "poly2", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "boolean": [{"name": "is_hole", "val": true}] } } ] } } ``` The value `MODE_POLY2D_ABSOLUTE` is the only supported value for `mode`. Absolute mode means that the values in `val` are interpreted as pixel coordinates (not as values relative to the first coordinate pair). ## Representing multi-polygons Multi-polygons are simply lists of polygons, so we describe these in a similar way with lists of `Poly2d` objects with the property `is_hole`. However, we also add one additional property `polygon_id` that determines which polygon a `Poly2d` object belongs to in the multi-polygon. Below follows an example of a multi-polygon with two polygons with one hole each. ```json { "object_data": { "poly2d": [ { "name": "poly1", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "text": [{"name": "polygon_id", "val": "1"}], "boolean": [{"name": "is_hole", "val": false}] } }, { "name": "poly2", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "text": [{"name": "polygon_id", "val": "1"}], "boolean": [{"name": "is_hole", "val": true}] } }, { "name": "poly3", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "text": [{"name": "polygon_id", "val": "2"}], "boolean": [{"name": "is_hole", "val": false}] } }, { "name": "poly4", "mode": "MODE_POLY2D_ABSOLUTE", "val": [...], "attributes": { "text": [{"name": "polygon_id", "val": "2"}], "boolean": [{"name": "is_hole", "val": true}] } } ] } } ``` The value `MODE_POLY2D_ABSOLUTE` is the only supported value for `mode`. Absolute mode means that the values in `val` are interpreted as pixel coordinates (not as values relative to the first coordinate pair). ## Representing curves :::hint{type="warning"} ### Caution The name of the interpolation method has changed from `interpolation-method` to `interpolation_method`. However, old annotations might still contain the old name ::: Curves are represented using the `poly2d` geometry and the interpolation method is specified as a text property in the following way. ```json { "poly2d": [ { "closed": false, "mode": "MODE_POLY2D_ABSOLUTE", "name": "curve-d633ca89", "val": [...], "attributes": { "text": [ { "name": "interpolation_method", "val": "natural-cubic-spline" } ] } } ] } ``` The value `MODE_POLY2D_ABSOLUTE` is the only supported value for `mode`. Absolute mode means that the values in `val` are interpreted as pixel coordinates (not as values relative to the first coordinate pair). The property `interpolation_method` is mandatory and determines how the nodes should be associated to each other. The following values are supported: - `natural-cubic-spline` - `catmull-rom-0.5` - `polyline` ## Representing 3D lanes A 3D lane is represented as two lines in 3D (`poly3d`), one to the right and the other to the left. The text property `lane_edge` determines whether the line is to the right or to the left. The lines will always have closed set to false. ```json { "object_data": { "poly3d": [ { "attributes": { "text": [ { "name": "lane_edge", "val": "left" }, { "name": "stream", "val": "lidar" } ] }, "closed": false, "name": "", "val": [ 1.2647494200238287, -51.51747573498745, -2.315540290283199, 1.0807419132566136, -48.91298533071834, -2.313640304199211, -0.0892715141237751, -34.705936676401016, -2.235569814758307, -0.4442893388935316, -29.60917111552865, -2.1894531147766174, -1.0952988968721313, -17.193981050037397, -2.1397902661132875 ] }, { "attributes": { "text": [ { "name": "lane_edge", "val": "right" }, { "name": "stream", "val": "lidar" } ] }, "closed": false, "name": "", "val": [ 1.5845765823868767, -51.49487958011918, -2.315540290283199, 1.4004322100638888, -48.888528958803036, -2.313640304199211, 0.23043085215069048, -34.68163859008775, -2.235569814758307, -0.12426061849402326, -29.589636067040036, -2.1894531147766174 ] } ] } } ``` ## Representing 2D points A 2D point is represented as a single point2d. Each `point2d` has an optional `point_class` attribute. For single points this may be ommited, but if set it must be equal to the `type` of the object. This attribute is reserved for future use on other point-based geometries. ```json { "openlabel": { ..., "frames": { "0": { "objects": { "a940239d-ff27-4480-8294-c482977a1b32": { "object_data": { "point2d": [ { "attributes": { "text": [ { "name": "point_class", "val": "APoint" }, { "name": "stream", "val": "stream1" } ] }, ... } ] } }, "e027e626-eb7a-4a8e-a9ae-083464e137d1": { "object_data": { "point2d": [ { "attributes": { "text": [ { "name": "stream", "val": "stream1" } ] }, .... } ] } } } } }, "metadata": {...}, "objects": { "a940239d-ff27-4480-8294-c482977a1b32": { ... "type": "APoint" }, "e027e626-eb7a-4a8e-a9ae-083464e137d1": { ... "type": "AnotherPoint" } }, ... } } ``` ## Representing groups of 2d points A group of points is used when multiple points refere to the same object. The attribute `point_class` is required for each of the points in the point group, and the `point_class` has to be different from the object type. The `point_class` value `"line_reference_point"` is reserved for future use cases. ## Representing Geometry Collections :::hint{type="info"} ### Note Introduced in `kognic_format_version` 2.2 ::: Related documenation for the task view [Geometry collections](docId\:U0O0ZfiuPRmbLHup6U4Mn). A collection of geometries as described in the link above will be represented as having a reserved relation as type `geometry_collection`. ```json { "openlabel": { "frames": { "0": { "objects": { "516b6045-87e8-40e4-a104-5eaa600e8e3a": { "object_data": { "bbox": [ { "name": "bbox-abc123", "val": [ ... ] } ] } }, "fe07e9cf-f42c-4b48-b4d8-bab75b7e9827": { "object_data": { "poly2d": [ { "name": "curve-abc123", "val": [ ... ] } ] } }, "329508b7-729c-4298-8141-f329dbc32ad0": { "object_data": { "poly2d": [ { "name": "curve-abc123", "val": [ ... ] } ] } }, "4c321584-0e88-4578-b0f0-b5e8c974244b": { "object_data": { "text": [ { "name": "lane", "val": "right" } ] } } } } }, "metadata": { "schema_version": "1.0.0", "kognic_format_version": "2.2", "uuid": "63698712-b18e-426b-9ad5-1b178cc29838" }, "objects": { "516b6045-87e8-40e4-a104-5eaa600e8e3a": { "name": "516b6045-87e8-40e4-a104-5eaa600e8e3a", "object_data": { ... }, "type": "some_bbox" }, "fe07e9cf-f42c-4b48-b4d8-bab75b7e9827": { "name": "fe07e9cf-f42c-4b48-b4d8-bab75b7e9827", "object_data": { ... }, "type": "some_line" }, "329508b7-729c-4298-8141-f329dbc32ad0": { "name": "329508b7-729c-4298-8141-f329dbc32ad0", "object_data": { ... }, "type": "some_line" }, "4c321584-0e88-4578-b0f0-b5e8c974244b": { "name": "4c321584-0e88-4578-b0f0-b5e8c974244b", "object_data": { ... }, "type": "some_collection" } }, "relations": { "0": { "name": "0", "rdf_objects": [ { "uid": "516b6045-87e8-40e4-a104-5eaa600e8e3a", "type": "object" }, { "uid": "fe07e9cf-f42c-4b48-b4d8-bab75b7e9827", "type": "object" }, { "uid": "329508b7-729c-4298-8141-f329dbc32ad0", "type": "object" } ], "rdf_subjects": [ { "uid": "4c321584-0e88-4578-b0f0-b5e8c974244b", "type": "object" } ], "type": "geometry_collection" } } } } ``` ## Representing 3D Segmentation Classifications are represented by run length encoding (RLE) the values. As an example, 11122222222000000000000 is converted into #3V1#8V2#13V0, where the number after character # defines the count of the repeated value, defined after character V. ```json "b088dcbf-c7fe-46a5-a71d-88ef49bdc107": { "object_data": { "binary": [ { "attributes": { "text": [ { "val": "lidar", "name": "stream" } ] }, "data_type": "", "encoding": "rle", "name": "labels", "val": "" } ] } } ```