[marker_radar_lidar_calibrator] Adding pair is not possible

[Sophie2216]

Hello!
Unfortunately, the marker radar lidar calibrator does not work for me. Hopefully, you can help me with the following problem:

I am trying to calibrate the lidar_top, which is a ouster_os2, with the lrr_front_ars548, which is a ARS548 3D radar. The initial TF tree is shown in the following screenshot:

For a better visualization, the TFs are located in the following way in 3D:

After the background extraction of the marker_radar_lidar_calibrator is finished, the corner reflector is placed into the environment and in the rviz2 visualization one can see the corner reflector as follows. Additionally, the blue dots are manually added to the marker_radar_lidar_calibrator view and showing the radar's detected objects according to the radar's topic.

A closer look is shown in the following two pictures, where it can be seen that the radars detected objects (blue) are not identical to the radar object_raw points (purple according your documentation of the marker_radar_lidar_calibrator). Several green marked lidar points (which are marker from the marker_radar_lidar_calibrator) also appear in the proximity of the corner reflector.


Since the marker_radar_lidar_calibrator finds lidar (green) and radar (purple) dots, the "Add lidar-radar pair" button was able to be pressed. Unfortunately, the pair is not established and the console has the following output:
[marker_radar_lidar_calibrator-2] [INFO] [1732291567.283530748] [marker_radar_lidar_calibrator]: lidarCallback [marker_radar_lidar_calibrator-2] [INFO] [1732291567.283549330] [marker_radar_lidar_calibrator]: There were no tracks [marker_radar_lidar_calibrator-2] [INFO] [1732291567.336945966] [marker_radar_lidar_calibrator]: lidarCallback [marker_radar_lidar_calibrator-2] [INFO] [1732291567.339657780] [marker_radar_lidar_calibrator]: Extracting foreground [marker_radar_lidar_calibrator-2] [INFO] [1732291567.339695228] [marker_radar_lidar_calibrator]: initial points: 131072 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.340580046] [marker_radar_lidar_calibrator]: cropped points: 129459 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.343981659] [marker_radar_lidar_calibrator]: voxel filtered points: 285 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.344404879] [marker_radar_lidar_calibrator]: tree filtered points: 12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.480623977] [marker_radar_lidar_calibrator]: ransac filtered points: 12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.480723653] [marker_radar_lidar_calibrator]: Cluster extraction input size: 12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.480730139] [marker_radar_lidar_calibrator]: found cluster of size: 12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.480738445] [marker_radar_lidar_calibrator]: Extracting lidar reflectors from clusters [marker_radar_lidar_calibrator-2] [INFO] [1732291567.480745983] [marker_radar_lidar_calibrator]: Colored clusters size=12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481355589] [marker_radar_lidar_calibrator]: Extracting foreground [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481374676] [marker_radar_lidar_calibrator]: initial points: 200 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481387904] [marker_radar_lidar_calibrator]: cropped points: 135 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481406840] [marker_radar_lidar_calibrator]: voxel filtered points: 127 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481496107] [marker_radar_lidar_calibrator]: tree filtered points: 70 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481580595] [marker_radar_lidar_calibrator]: Cluster extraction input size: 70 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481588946] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481593272] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481597285] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481600920] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481604756] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481608270] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481611837] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481615115] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481618768] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481622034] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481625137] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481628448] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481631796] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481634958] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481638471] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481642061] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481645473] [marker_radar_lidar_calibrator]: found cluster of size: 4 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481649190] [marker_radar_lidar_calibrator]: found cluster of size: 1 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481652909] [marker_radar_lidar_calibrator]: found cluster of size: 1 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481659360] [marker_radar_lidar_calibrator]: Extracting radar reflectors from clusters [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481663864] [marker_radar_lidar_calibrator]: Radar reflector id=0 size=4 center: x=48.80 y=31.76 z=1.38 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481670877] [marker_radar_lidar_calibrator]: Radar reflector id=1 size=4 center: x=46.23 y=-12.88 z=1.37 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481676019] [marker_radar_lidar_calibrator]: Radar reflector id=2 size=4 center: x=46.57 y=-0.98 z=0.97 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481694995] [marker_radar_lidar_calibrator]: Radar reflector id=3 size=4 center: x=43.78 y=0.54 z=0.74 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481700286] [marker_radar_lidar_calibrator]: Radar reflector id=4 size=4 center: x=43.71 y=31.40 z=1.52 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481705257] [marker_radar_lidar_calibrator]: Radar reflector id=5 size=4 center: x=38.28 y=-7.61 z=0.98 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481710326] [marker_radar_lidar_calibrator]: Radar reflector id=6 size=4 center: x=32.35 y=13.40 z=1.68 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481715132] [marker_radar_lidar_calibrator]: Radar reflector id=7 size=4 center: x=30.22 y=-4.73 z=0.95 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481719935] [marker_radar_lidar_calibrator]: Radar reflector id=8 size=4 center: x=24.12 y=-3.04 z=0.46 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481724275] [marker_radar_lidar_calibrator]: Radar reflector id=9 size=4 center: x=20.25 y=-3.39 z=0.05 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481728954] [marker_radar_lidar_calibrator]: Radar reflector id=10 size=4 center: x=13.09 y=-10.34 z=1.01 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481733534] [marker_radar_lidar_calibrator]: Radar reflector id=11 size=4 center: x=10.21 y=-0.30 z=-0.02 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481738467] [marker_radar_lidar_calibrator]: Radar reflector id=12 size=4 center: x=9.80 y=-2.76 z=0.47 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481743346] [marker_radar_lidar_calibrator]: Radar reflector id=13 size=4 center: x=10.57 y=-5.40 z=0.13 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481748367] [marker_radar_lidar_calibrator]: Radar reflector id=14 size=4 center: x=7.16 y=4.29 z=-0.23 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481752716] [marker_radar_lidar_calibrator]: Radar reflector id=15 size=4 center: x=4.87 y=-2.68 z=0.95 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481756941] [marker_radar_lidar_calibrator]: Radar reflector id=16 size=4 center: x=28.56 y=13.86 z=1.12 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481761633] [marker_radar_lidar_calibrator]: Radar reflector id=17 size=1 center: x=19.72 y=13.63 z=0.95 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.481766340] [marker_radar_lidar_calibrator]: Radar reflector id=18 size=1 center: x=15.98 y=-15.70 z=0.37 [marker_radar_lidar_calibrator-2] [INFO] [1732291567.482007915] [marker_radar_lidar_calibrator]: Lidar detections: 0 Radar detections: 19 Matches: 0 Active tracks; 0 Converged tracks: 0
So it seems that radar detections are found but no lidar detections according to the console output although in the rviz2 visualization there are green marked lidar points.

So, why is it not possible to add the lidar-radar pair?

[knzo25]

@Sophie2216
Hi, thank for using our tools.

Regarding the ARS548, we support it in this branch. Are you using it?
In a little unrelated topic, are you using our driver for that device?

Until the related PR is merged it is difficult to give proper support. That being said, if you provide us some rosbag, we can at least have a look a it !

[Sophie2216]

@knzo25
Hi,
Currently, we are using the tier4/universe. So, thanks for redirecting us to the "feature/marker_radar_lidar_calibrator_support_radars_and_transformation_algorithms" branch. We will try this one.
Yes, we are using your driver for that device.

I would like to send you the rosbag via email. Would this be possible?

[knzo25]

Thanks for the confirmation, the before mentioned PR used our driver so I can not really give any sort of guarantees otherwise.
My mail is kenzo.lobos at tier4.jp

[knzo25]

@Sophie2216
Sorry, I have been away for an experiment. As soon as I finish catching up with work I will take a look

[Sophie2216]

@knzo25
No problem!

We have tried to do the lidar-radar calibration by using the "feature/marker_radar_lidar_calibrator_support_radars_and_transformation_algorithms" branch. Unfortunately, it didn't work. The default project was selected in combination with the marker_radar_lidar_calibrator and the following variables were inserted in the launcher configuration:

We thought that the radar's scan_raw topic may fit best from the following possible topics of the ARS548 radar:
/sensing/radar/lrr_front_ars548/detection_points
/sensing/radar/lrr_front_ars548/detections
/sensing/radar/lrr_front_ars548/marker_array
/sensing/radar/lrr_front_ars548/nebula_packets
/sensing/radar/lrr_front_ars548/object_points
/sensing/radar/lrr_front_ars548/objects
/sensing/radar/lrr_front_ars548/objects_raw
/sensing/radar/lrr_front_ars548/objects_raw_filtered
/sensing/radar/lrr_front_ars548/output/radar_detected_objects
/sensing/radar/lrr_front_ars548/output/radar_tracked_objects
/sensing/radar/lrr_front_ars548/scan_raw

The sensor_calibration_manager window, RViz, and the metrics plotter appeared, but the 'Calibrate' button was not available and in the terminal the following lines showed up:
{ade}-adm@innb0:~/WARE/calibration_tools_ws$ros2 run sensor_calibration_manager sensor_calibration_manager
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-ware'
INFO:root:on_selected_calibrator: project_name=default_project calibrator_name=marker_radar_lidar_calibrator
INFO:root:Reading xml from: /home/ware/WARE/calibration_tools_ws/install/sensor_calibration_manager/share/sensor_calibration_manager/launch/default_project/marker_radar_lidar_calibrator.launch.xml
INFO:root:Closing calibrator selector view
^[[AINFO:root:Launching calibrator with the following command: ['ros2', 'launch', '/home/ware/WARE/calibration_tools_ws/install/sensor_calibration_manager/share/sensor_calibration_manager/launch/default_project/marker_radar_lidar_calibrator.launch.xml', 'radar_optimization_frame:=base_link', 'radar_frame:=lrr_front_ars548', 'lidar_frame:=lidar_top/os_lidar', 'msg_type:=radar_scan', 'transformation_type:=svd_3d', 'input_radar_msg:=/sensing/radar/lrr_front_ars548/scan_raw', 'input_lidar_pointcloud:=/sensing/lidar/top/points', 'rviz:=true']
[INFO] [launch]: All log files can be found below /home/ware/.ros/log/2024-12-13-14-15-05-821125-ware2-11976
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [static_transform_publisher-1]: process started with pid [11998]
[INFO] [marker_radar_lidar_calibrator-2]: process started with pid [12000]
[INFO] [calibrator_ui_node.py-3]: process started with pid [12002]
[INFO] [metrics_plotter_node.py-4]: process started with pid [12004]
[INFO] [rviz2-5]: process started with pid [12006]
[static_transform_publisher-1] [WARN] [1734095706.152362657] []: Old-style arguments are deprecated; see --help for new-style arguments
[static_transform_publisher-1] [INFO] [1734095706.162631928] [lidar_broadcaster]: Spinning until stopped - publishing transform
[static_transform_publisher-1] translation: ('0.000000', '0.000000', '0.000000')
[static_transform_publisher-1] rotation: ('0.000000', '0.000000', '0.000000', '1.000000')
[static_transform_publisher-1] from 'lrr_front_ars548' to 'radar_frame'
[marker_radar_lidar_calibrator-2] ERROR: flag 'logtostderr' was defined more than once (in files './src/logging.cc' and '/home/ware/WARE/autoware/src/universe/external/glog/src/logging.cc').
[ERROR] [marker_radar_lidar_calibrator-2]: process has died [pid 12000, exit code 1, cmd '/home/ware/WARE/calibration_tools_ws/install/marker_radar_lidar_calibrator/lib/marker_radar_lidar_calibrator/marker_radar_lidar_calibrator --ros-args -r __node:=marker_radar_lidar_calibrator -r __ns:=/ --params-file /tmp/launch_params_b5w3go4o --params-file /tmp/launch_params_6kcf_vkz --params-file /tmp/launch_params_qb1f3nna --params-file /tmp/launch_params_nhoc1w32 --params-file /tmp/launch_params_d39u93di --params-file /tmp/launch_params_l3uild_r --params-file /tmp/launch_params_6mek7k99 --params-file /tmp/launch_params___den0ar --params-file /tmp/launch_params_edolvfxj --params-file /tmp/launch_params_imn0r8is --params-file /tmp/launch_params_u06dit3h --params-file /tmp/launch_params_hpoke5d2 --params-file /tmp/launch_params_o3fuzqbh --params-file /tmp/launch_params_g4_rq1wu --params-file /tmp/launch_params_5v02qdqq --params-file /tmp/launch_params_xirtwwkv --params-file /tmp/launch_params_s0dy57ls -r extrinsic_calibration:=calibrate_radar_lidar -r input_lidar_pointcloud:=/sensing/lidar/top/points -r input_radar_msg:=/sensing/radar/lrr_front_ars548/scan_raw'].

The following xml-file (I had to upload it as a txt-file since xml is not a supported file format for uploading) was used for launching the ARS548:
lrr_front_ars548.launch.txt

I hope that you can help with this amount of information to solve the issue such that we can calibrate with the marker_radar_lidar_calibrator.

Thanks for your help!

[Sophie2216]

Hi!
Do you have any idea why the marker_radar_lidar_calibrator from branch "feature/marker_radar_lidar_calibrator_support_radars_and_transformation_algorithms" is not working for us?