Exwayz Colorizaton algorithm performs colorization on LiDAR frames from one or multiple cameras.

See exwayz_colorization.exe for usage information.

Below is a sample colorization configuration file.

pc_colorization:
    # [optional, default=true] Filter the output frames keeping only points 
    # falling in the FoV of at least 1 camera
    filter_pc: true
    # [optional, default=false] Prevents coloring points being occluded by others
    filter_on_depth: false
    # [optional, default=0.500(s)] Maximum time offset between a LiDAR frame and
    # and image used to colorize it
    dt_max: 0.5
    # [optional, default=0.5] Extra brightness (multiplier on r, g, b: if set to
    # 0.3, (r, g, b) is multiplied x1.3)
    extra_brightness: 0.5

    # [required] Set of input cameras
    cameras:
        # [required] Camera name
        - name: perspective_camera
          # [required] Camera type. Supported types are
          # * 0: pinhole
          # * 1: equirectangular
          type: 0
          # [required for pinhole] Path to camera intrisic calibration (camera 
          # matrix and distorsion coefficients)
          intrinsic: <path/to/>intrinsics.xml
          # [required] Path to the LiDAR -> camera extrinsic calibration
          extrinsic: <path/to/>extrinsics.txt
          # [required] Camera source type
          source:
              # Path to a directory of images
              directory: ./images
              # [optional, default=0.0(s)] Time offset of the camera wrt LiDAR timestamps
              time_offset: 0.0

        - name: equirectangular_camera
          type: 1
          extrinsic: extrinsics.yaml
          # [optional] Path to a mask for preventing colorizing using a given 
          # part of the images
          mask: mask.png
          source:
              # Path to a video file.
              video: ./panoramic.mp4
              time_offset: -0.5

Explanation

filter_pc

filer_on_depth

dt_max

extra_brightness

cameras

Intrinsic supported formats

Below are examples of the supported intrinsic calibration data. The required informations are

• The camera matrix

$$ \begin{pmatrix}f_x & 0 & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1\end{pmatrix} $$

• The distorsion cofficients

$$ \begin{pmatrix}k_1 & k_2 & p_1& p_2&k_3\end{pmatrix} $$

Refer to https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html if needed.

XML