Incorrect Bounding Box Position, Rotation, and Size with Actors Having Roll and Pitch in CARLA 9.14

[aiglesias-vicom]

Hello,

I am using CARLA version 9.14 on Ubuntu, and I am working on labeling falling obstacles in my simulation. To achieve this, I recorded obstacles falling onto the road and developed a custom labeling tool to annotate the objects. This approach has worked perfectly until I began introducing actors with both pitch and roll.

In the following image, you can see a falling static.prop.box01 and its corresponding bounding box. There is a clear discrepancy in the position, rotation, and size of the bounding box when compared to the actual object.

I am labeling the actors' bounding boxes using the world coordinate system and also storing the point cloud data in the same system. The following code snippet is used to retrieve the bounding box of an actor:

# Actor bbox
obj_bbox = obj.bounding_box
# Get bbox size
bbox_size = [obj_bbox.extent.x, obj_bbox.extent.y, obj_bbox.extent.z]

# BBox coords in the object coordinate system
cords = create_bb_points(bbox_size)
bb_transform = carla.Transform(obj_bbox.location)
bb_obj_matrix = get_matrix(bb_transform)

# Object transform in world coordinate system
obj_transform = obj.get_transform()
obj_world_matrix = get_matrix(obj_transform)
bb_world_matrix = np.dot(obj_world_matrix, bb_obj_matrix)

# Vertices in the (left handed) world coordinate system
world_cords = np.dot(rh2lf_mat_4x4, np.dot(bb_world_matrix, np.transpose(cords)))

This code is based on the function vehicle_to_world from the repo. Additionally, I use other functions from the same repository to help with the annotation process:

def create_bb_points(vehicle):
    cords = np.zeros((8, 4))
    extent = vehicle.bounding_box.extent
    cords[0, :] = np.array([extent.x, extent.y, -extent.z, 1])
    cords[1, :] = np.array([-extent.x, extent.y, -extent.z, 1])
    cords[2, :] = np.array([-extent.x, -extent.y, -extent.z, 1])
    cords[3, :] = np.array([extent.x, -extent.y, -extent.z, 1])
    cords[4, :] = np.array([extent.x, extent.y, extent.z, 1])
    cords[5, :] = np.array([-extent.x, extent.y, extent.z, 1])
    cords[6, :] = np.array([-extent.x, -extent.y, extent.z, 1])
    cords[7, :] = np.array([extent.x, -extent.y, extent.z, 1])
    return cords

### Get transformation matrix from carla.Transform object
def get_matrix(transform):
    rotation = transform.rotation
    location = transform.location
    c_y = np.cos(np.radians(rotation.yaw))
    s_y = np.sin(np.radians(rotation.yaw))
    c_r = np.cos(np.radians(rotation.roll))
    s_r = np.sin(np.radians(rotation.roll))
    c_p = np.cos(np.radians(rotation.pitch))
    s_p = np.sin(np.radians(rotation.pitch))
    matrix = np.matrix(np.identity(4))
    matrix[0, 3] = location.x
    matrix[1, 3] = location.y
    matrix[2, 3] = location.z
    matrix[0, 0] = c_p * c_y
    matrix[0, 1] = c_y * s_p * s_r - s_y * c_r
    matrix[0, 2] = -c_y * s_p * c_r - s_y * s_r
    matrix[1, 0] = s_y * c_p
    matrix[1, 1] = s_y * s_p * s_r + c_y * c_r
    matrix[1, 2] = -s_y * s_p * c_r + c_y * s_r


    matrix[2, 0] = s_p
    matrix[2, 1] = -c_p * s_r
    matrix[2, 2] = c_p * c_r
    return matrix    

As mentioned, I had no issues with this method until I started testing scenes with objects that have roll and pitch. To help reproduce the issue, I have provided the CARLA recording:

falling_obstacle.log

I have also included the labeled data in OpenLABEL format. After weeks of testing and debugging, I suspect the error is related to how CARLA calculates bounding boxes, as well as the position and rotation of actors when roll and pitch are involved.

Any guidance or suggestions on resolving this issue would be greatly appreciated.

Thank you for your time and assistance.