OpenLABEL Format

openlabel is a standardized annotation format developed by asam https //www asam net/ it comes with in depth documentation https //www asam net/index php?eid=dumpfile\&t=f\&f=4566\&token=9d976f840af04adee33b9f85aa3c22f2de4968dd of the format as well as a json schema https //openlabel asam net/v1 0 0/schema/openlabel json schema json , which can be used to ensure that the format is valid even though openlabel is a strict format there is still some room for interpretation in this section we try to clarify some of these parts and explain the choices that we have made within the standard rotation of cuboids the rotation is such that the y axis is facing forwards, with a rotation order of xyz this means that a cuboid with a heading (yaw) equal to 0 is aligned with the y axis in the positive direction along the axis this is somewhat different compared to the iso 8855 https //www sis se/api/document/preview/914200/# text=this%20international%20standard%20defines%20the,to%20multi%2dunit%20vehicle%20combinations standard, where the forward direction is along the x axis conversion to iso 8855 can then be done by applying a rotation around the z axis and changing sx and sy in the following way import math from typing import list from scipy spatial transform import rotation def convert to iso8855(val list\[float]) > list\[float] """ converts cuboid values to iso 8855 """ \[x, y, z, qx, qy, qz, qw, sx, sy, sz] = val rotation 1 = rotation from quat(\[qx, qy, qz, qw]) rotation 2 = rotation from rotvec(\[0, 0, math pi / 2]) rot object = rotation 1 rotation 2 \[qx, qy, qz, qw] = rot object as quat() return \[x, y, z, qx, qy, qz, qw, sy, sx, sz] non sequences are sequences with one frame due to reasons of simplicity we have made the choice to treat non sequences in the same way as sequences this means that non sequences are represented as a sequence with only one frame only data such as name and type are defined in the top level element keys all other information is stored under frames, see example below { "objects" { "0" { "name" "car 0", "type" "car" } }, "frames" { "0" { "objects" { "0" {"object data" { }} } } } } stream is just another text property the stream property is used to indicate which stream/sensor/source that the geometry och property was annotated in for example here is an object with a point that has been annotated in a stream with the name camera note that all corresponding attributes for the geometry have also been annotated in the same stream { "object data" { "point2d" \[ { "name" "point 4d2d325f", "val" \[300 5300, 286 4396], "attributes" { "text" \[ {"name" "stream", "val" "camera"}, {"name" "color", "val" "black"} ] } } ] } } relations we consider two types of relations; unidirectional relations between two objects and group relations in addition to these, there is a need to represent false relations, i e relation properties that are not actually pointers to other objects but rather take values such as inconclusive , nothing or unclear relations are unidirectional relations are unidirectional, meaning that if an object, object1 , has a relation to another object, object2 , it does not mean that object2 has a relation to object1 below follows an example where car 0 is following car 1 and it is unclear whether car 2 is following another car or not { "objects" { "0" {"name" "car 0", "type" "car"}, "1" {"name" "car 1", "type" "car"}, "2" {"name" "car 2", "type" "car"} }, "relations" { "0" { "name" "0", "type" "isfollowing", "rdf subjects" \[{"type" "object", "uid" "0"}], "rdf objects" \[{"type" "object", "uid" "1"}] } } } actions are used to represent false relations in the kognic platform, there is support for assigning values to relations that are not actually references to other objects examples are inconclusive and nothing actions are used to represent these in the following way, where the name of the action determines the value and the type determines the property name { "objects" { "0" {"name" "lane 0", "type" "lane"} }, "relations" { "0" { "name" "0", "type" "issubjectofaction", "rdf subjects" \[{"type" "object", "uid" "0"}], "rdf objects" \[{"type" "action", "uid" "0"}] } }, "actions" { "0" {"name" "nothing", "type" "is pulling or pushing"} } } stream specific relations if a relation is stream specific, there will be a property stream relations denoting which stream the list of relations belong to { // frames 0 // "frame properties" { "streams" { "camera front" { "description" null, "stream properties" { "stream relations" { "1" {} } } } } }, "relations" { "0" {} } } representing polygons polygons are described by a list of poly2d objects in openlabel one of these represents the exterior while the others represent potential holes and this is determined by the boolean property is hole below follows an example of a polygon with one hole { "object data" { "poly2d" \[ { "name" "poly1", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "boolean" \[{"name" "is hole", "val" false}] } }, { "name" "poly2", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "boolean" \[{"name" "is hole", "val" true}] } } ] } } the value mode poly2d absolute is the only supported value for mode absolute mode means that the values in val are interpreted as pixel coordinates (not as values relative to the first coordinate pair) representing multi polygons multi polygons are simply lists of polygons, so we describe these in a similar way with lists of poly2d objects with the property is hole however, we also add one additional property polygon id that determines which polygon a poly2d object belongs to in the multi polygon below follows an example of a multi polygon with two polygons with one hole each { "object data" { "poly2d" \[ { "name" "poly1", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "text" \[{"name" "polygon id", "val" "1"}], "boolean" \[{"name" "is hole", "val" false}] } }, { "name" "poly2", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "text" \[{"name" "polygon id", "val" "1"}], "boolean" \[{"name" "is hole", "val" true}] } }, { "name" "poly3", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "text" \[{"name" "polygon id", "val" "2"}], "boolean" \[{"name" "is hole", "val" false}] } }, { "name" "poly4", "mode" "mode poly2d absolute", "val" \[ ], "attributes" { "text" \[{"name" "polygon id", "val" "2"}], "boolean" \[{"name" "is hole", "val" true}] } } ] } } the value mode poly2d absolute is the only supported value for mode absolute mode means that the values in val are interpreted as pixel coordinates (not as values relative to the first coordinate pair) representing curves caution the name of the interpolation method has changed from interpolation method to interpolation method however, old annotations might still contain the old name curves are represented using the poly2d geometry and the interpolation method is specified as a text property in the following way { "poly2d" \[ { "closed" false, "mode" "mode poly2d absolute", "name" "curve d633ca89", "val" \[ ], "attributes" { "text" \[ { "name" "interpolation method", "val" "natural cubic spline" } ] } } ] } the value mode poly2d absolute is the only supported value for mode absolute mode means that the values in val are interpreted as pixel coordinates (not as values relative to the first coordinate pair) the property interpolation method is mandatory and determines how the nodes should be associated to each other the following values are supported natural cubic spline catmull rom 0 5 polyline representing 3d lanes a 3d lane is represented as two lines in 3d ( poly3d ), one to the right and the other to the left the text property lane edge determines whether the line is to the right or to the left the lines will always have closed set to false { "object data" { "poly3d" \[ { "attributes" { "text" \[ { "name" "lane edge", "val" "left" }, { "name" "stream", "val" "lidar" } ] }, "closed" false, "name" "", "val" \[ 1 2647494200238287, 51 51747573498745, 2 315540290283199, 1 0807419132566136, 48 91298533071834, 2 313640304199211, 0 0892715141237751, 34 705936676401016, 2 235569814758307, 0 4442893388935316, 29 60917111552865, 2 1894531147766174, 1 0952988968721313, 17 193981050037397, 2 1397902661132875 ] }, { "attributes" { "text" \[ { "name" "lane edge", "val" "right" }, { "name" "stream", "val" "lidar" } ] }, "closed" false, "name" "", "val" \[ 1 5845765823868767, 51 49487958011918, 2 315540290283199, 1 4004322100638888, 48 888528958803036, 2 313640304199211, 0 23043085215069048, 34 68163859008775, 2 235569814758307, 0 12426061849402326, 29 589636067040036, 2 1894531147766174 ] } ] } } representing 2d points a 2d point is represented as a single point2d each point2d has an optional point class attribute for single points this may be ommited, but if set it must be equal to the type of the object this attribute is reserved for future use on other point based geometries { "openlabel" { , "frames" { "0" { "objects" { "a940239d ff27 4480 8294 c482977a1b32" { "object data" { "point2d" \[ { "attributes" { "text" \[ { "name" "point class", "val" "apoint" }, { "name" "stream", "val" "stream1" } ] }, } ] } }, "e027e626 eb7a 4a8e a9ae 083464e137d1" { "object data" { "point2d" \[ { "attributes" { "text" \[ { "name" "stream", "val" "stream1" } ] }, } ] } } } } }, "metadata" { }, "objects" { "a940239d ff27 4480 8294 c482977a1b32" { "type" "apoint" }, "e027e626 eb7a 4a8e a9ae 083464e137d1" { "type" "anotherpoint" } }, } } representing groups of 2d points a group of points is used when multiple points refere to the same object the attribute point class is required for each of the points in the point group, and the point class has to be different from the object type the point class value "line reference point" is reserved for future use cases representing geometry collections note introduced in kognic format version 2 2 related documenation for the task view class groups docid\ u0o0zfiuprmblhup6u4mn a collection of geometries as described in the link above will be represented as having a reserved relation as type geometry collection { "openlabel" { "frames" { "0" { "objects" { "516b6045 87e8 40e4 a104 5eaa600e8e3a" { "object data" { "bbox" \[ { "name" "bbox abc123", "val" \[ ] } ] } }, "fe07e9cf f42c 4b48 b4d8 bab75b7e9827" { "object data" { "poly2d" \[ { "name" "curve abc123", "val" \[ ] } ] } }, "329508b7 729c 4298 8141 f329dbc32ad0" { "object data" { "poly2d" \[ { "name" "curve abc123", "val" \[ ] } ] } }, "4c321584 0e88 4578 b0f0 b5e8c974244b" { "object data" { "text" \[ { "name" "lane", "val" "right" } ] } } } } }, "metadata" { "schema version" "1 0 0", "kognic format version" "2 2", "uuid" "63698712 b18e 426b 9ad5 1b178cc29838" }, "objects" { "516b6045 87e8 40e4 a104 5eaa600e8e3a" { "name" "516b6045 87e8 40e4 a104 5eaa600e8e3a", "object data" { }, "type" "some bbox" }, "fe07e9cf f42c 4b48 b4d8 bab75b7e9827" { "name" "fe07e9cf f42c 4b48 b4d8 bab75b7e9827", "object data" { }, "type" "some line" }, "329508b7 729c 4298 8141 f329dbc32ad0" { "name" "329508b7 729c 4298 8141 f329dbc32ad0", "object data" { }, "type" "some line" }, "4c321584 0e88 4578 b0f0 b5e8c974244b" { "name" "4c321584 0e88 4578 b0f0 b5e8c974244b", "object data" { }, "type" "some collection" } }, "relations" { "0" { "name" "0", "rdf objects" \[ { "uid" "516b6045 87e8 40e4 a104 5eaa600e8e3a", "type" "object" }, { "uid" "fe07e9cf f42c 4b48 b4d8 bab75b7e9827", "type" "object" }, { "uid" "329508b7 729c 4298 8141 f329dbc32ad0", "type" "object" } ], "rdf subjects" \[ { "uid" "4c321584 0e88 4578 b0f0 b5e8c974244b", "type" "object" } ], "type" "geometry collection" } } } } representing 3d segmentation classifications are represented by run length encoding (rle) the values as an example, 11122222222000000000000 is converted into #3v1#8v2#13v0, where the number after character # defines the count of the repeated value, defined after character v "b088dcbf c7fe 46a5 a71d 88ef49bdc107" { "object data" { "binary" \[ { "attributes" { "text" \[ { "val" "lidar", "name" "stream" } ] }, "data type" "", "encoding" "rle", "name" "labels", "val" "\<rle string>" } ] } }