Custom Camera Calibrations

note this feature is new in version 1 8 0 of kognic io and some parts require optional dependencies run pip install kognic io\[wasm] to install it if your calibration is not in the list of supported standard camera calibrations, you can provide a custom calibration the table below show the attributes of the customcameracalibration object the wasm base64 attribute is a base64 encoded string of the webassembly module that implements the calibration the test cases attribute is a list of testcase objects that are used to validate the calibration during creation it is recommended to provide a few test cases to make sure that the calibration is correct key value parameters rotation quaternion a rotationquaternion object w , x , y , z position a position object x , y , z image width integer na image height integer na wasm base64 string na test cases a list of testcase objects point3d, point2d the customcameracalibration object can conveniently be instantiated directly from the binary or a wasm file calibration file = customcameracalibration from file("/path/to/calibration wasm", ) calibration binary = customcameracalibration from bytes(b" ", ) we provide a set of utilities that will make it easier to work with the webassembly module more specifically we provide validation code and compilation code from a few different languages to webassembly the code is available both as python functions and via the kognicutil cli the webassembly module the webassembly must follow a strict interface where the module exports a function called project point to image the function must take three arguments of type float64 and return two values of type float64 thus, the webassembly text representation of this interface is func (param f64 f64 f64) (result f64 f64) the three arguments are the x, y and z coordinates of the 3d point the two return values are the x and y coordinates of the projected point in the image plane webassembly doesn't support multiple return values by default but this can be enabled with the multi value https //github com/webassembly/multi value proposal if the point is not within the field of view, the function should return nan https //en wikipedia org/wiki/nan for both the x and y coordinates validation note this requires wasmtime to be installed which is an optional of dependency kognic io run pip install kognic io\[wasm] to install it we provide validation code both as python functions and via the kognicutil cli we validate things such as but not limited to the module can be loaded the function exists and has the correct signature that a point can be projected using the module that points are projected correctly if test cases are provided in python there are three different ways to validate a calibration import kognic io tools calibration validation as wasm validation from kognic io model calibration camera custom camera calibration import customcameracalibration, point2d, point3d, testcase test cases = \[ testcase( point3d=point3d(x=1 0, y=2 0, z=3 0), point2d=point2d(x=2 0, y=5 6) ), testcase( point3d=point3d(x=1 0, y=1 0, z= 1 0), point2d=point2d(x=float("nan"), y=float("nan")) # point is outside field of view ) ] wasm file = "/path/to/calibration wasm" calibration = customcameracalibration from bytes(wasm file, test cases=test cases, ) wasm bytes = calibration to bytes() \# validate the calibration object wasm validation validate custom camera calibration(calibration, test cases=test cases) \# validate the wasm file wasm validation validate wasm file(wasm file, test cases=test cases) \# validate the wasm binary wasm validation validate wasm bytes(wasm bytes, test cases=test cases) the kognicutil cli can be used as follows kognicutil wasm validate calibration wasm compilation caution rust 1 82 0 removes support for the multivalue feature target (returning multiple values from a function) since this feature is currently needed for custom camera calibrations to work, the rust and/or cargo version needs to be pinned to < 1 82 0 note it is recommended to keep the wasm file as small as possible try to avoid dependencies that are not needed for example, it may be preferred to implement some mathematical functions yourself instead of using the standard library as stated above the webassembly module must follow a strict interface and compilation requires the multi value proposal we provide a set of utilities that will make it easier to compile the webassembly module from a few languages, see table below language target compilation tool required version rust rs rustc < 1 82 0 rust (cargo) cargo toml cargo < 1 82 0 c++ cc, cpp emscripten n/a c c emscripten n/a the utilities are available both as python functions and via the kognicutil cli from python, you can compile the module with from kognic io tools calibration compilation import compile to wasm wasm binary = compile to wasm("path/to/source") the returned binary can then be used to create a customcameracalibration object if the output wasm parameter is passed, the binary will be saved to the specified path the kognicutil cli can be used as follows kognicutil wasm compile path/to/source path/to/output wasm note that, validation is run by default after compilation this can be disabled with the skip validation flag calibration parameters have to be embedded in the binary so that they can be used by the webassembly module try to pre compute as much as possible to increase the speed of the projection function at runtime below follows examples of a simplified version of the pinhole calibration in a few different languages example rust note rust needs to be installed with the wasm32 wasi target for this install rust according to the instructions here https //www rust lang org/tools/install and then add the target with rustup target add wasm32 wasi a rust file https //github com/annotell/kognic io examples python/tree/master/examples/calibration/custom/pinhole rs can be compiled with kognicutil wasm compile path/to/source rs path/to/output wasm note that panics are not supported and compilation will fail if the code contains it rust with cargo note rust and cargo need to be installed with the wasm32 wasi target for this install rust and cargo according to the instructions here https //doc rust lang org/cargo/getting started/installation html and then add the target with rustup target add wasm32 wasi a rust module with cargo https //github com/annotell/kognic io examples python/tree/master/examples/calibration/custom/pinhole rust cargo can be compiled with kognicutil wasm compile path/to/source/cargo toml path/to/output wasm note that it is important to specify that the library is a cdylib and it is also recommended to set strip = true to reduce the size of the webassembly module this is done by adding the following to the cargo toml file \[lib] crate type = \["cdylib"] \[profile release] strip = true example c++ note emscripten needs to be installed for this, which can be done according to the instructions here https //emscripten org/docs/getting started/downloads html a c++ file https //github com/annotell/kognic io examples python/tree/master/examples/calibration/custom/pinhole cc can be compiled with kognicutil wasm compile path/to/source cc path/to/output wasm or with kognicutil wasm compile path/to/source cpp path/to/output wasm example c note emscripten needs to be installed for this, which can be done according to the instructions here https //emscripten org/docs/getting started/downloads html a c file https //github com/annotell/kognic io examples python/tree/master/examples/calibration/custom/pinhole c can be compiled with kognicutil wasm compile path/to/source c path/to/output wasm