Visualizing camera trajectory in Open3D

[maheshkkumar]

Hi @AlexanderFabisch,

I am trying to use pytransform3d for some visualizations. So, I was wondering if there is a particular example/tutorial in the documentation related to this image in README for visualizing the camera trajectory in Open3D.

Thank you!

[AlexanderFabisch]

Hi @maheshkkumar,

there is no magic. Figure.plot_camera is all you need to read. If the explanation is not enough, let me know how to improve it.

The specific example that you refer to has been generated from the output of the software meshroom with this code:

import sys
import json
import numpy as np
import pytransform3d.transformations as pt
import pytransform3d.camera as pc
import pytransform3d.visualizer as pv


mesh_filename = sys.argv[-2]
camera_filename = sys.argv[-1]

with open(camera_filename, "r") as f:
    cameras = json.load(f)

camera_poses = cameras["poses"]
camera_intrinsics = cameras["intrinsics"][0]

px_focal_length = float(camera_intrinsics["pxFocalLength"])
px_principal_point_x = float(camera_intrinsics["principalPoint"][0])
px_principal_point_y = float(camera_intrinsics["principalPoint"][1])
M = np.array([
    [px_focal_length, 0, px_principal_point_x],
    [0, px_focal_length, px_principal_point_y],
    [0, 0, 1]
])
sensor_size = (float(camera_intrinsics["width"]), float(camera_intrinsics["height"]))

transformation_matrices = np.empty((len(camera_poses), 4, 4))
for i, camera_pose in enumerate(camera_poses):
    R = np.array(list(map(float, camera_pose["pose"]["transform"]["rotation"]))).reshape(3, 3)
    p = np.array(list(map(float, camera_pose["pose"]["transform"]["center"])))
    transformation_matrices[i] = pt.transform_from(R=R, p=p)


fig = pv.figure()
fig.plot_mesh(mesh_filename)
for pose in transformation_matrices:
    fig.plot_transform(A2B=pose, s=0.1)
    fig.plot_camera(M=M, cam2world=pose, virtual_image_distance=0.1, sensor_size=sensor_size)
fig.show()

[maheshkkumar]

Thanks for the quick reply!

[nnop]

Do you know how to implement this with open3d?

[AlexanderFabisch]

pytransform3d wraps Open3d in this implementation. Take a look at the source code of the visualizer module to see how open3d is used.

[Mechazo11]

@AlexanderFabisch if the sensor's true dimensions are not known, would it affect the outcome of the 3d plot if we just use the default values (sensor_size = np.array([0.036, 0.024])) from this example https://dfki-ric.github.io/pytransform3d/_auto_examples/plots/plot_camera_3d.html#sphx-glr-auto-examples-plots-plot-camera-3d-py ?

[AlexanderFabisch]

It will only affect the size of the virtual image: https://en.wikipedia.org/wiki/Virtual_image

This shouldn't really matter if you only use it for visualization.