ZED vertical mount, projection is wrong

[erminkev1]

Hello!

I've been experimenting with a ZED camera and a Ouster Lidar. The positioning of the sensors is such that lidar faces forward, while the camera faces to the right of it. The camera is also vertically. I've calculated the transformation matrix, and accounted for this setup in the launch file configuration, however the projection of LiDAR points onto my image from ZED is quite poor. My suspicion was first that LiDAR and ZED get initialized in different coordinate systems, and trying for variations of them did not make much difference. I was even thinking that the intermediate step of using RangeImage and its LASER_FRAME had to do something with this as well, but I am not quite sure.

Please note that, when I mounted both of my sensors facing forward, I could easily adjust the parameters to get an output that I want, but this new setup breaks constantly.

Are there any suggestions that someone may provide regarding this kind of setup.