visible object can't found in event.instance_detections2D or frame

[cghAndy]

I search all objects in event.metadata["objects"]，to get the visible objects' bbox
But I found that many visible object can't found in event.instance_detections2D or frame
here's an example
Bed and CD are visible.
*visible means metadata[object]['visible'] = True

[Lucaweihs]

Hi @cghAndy,

Thanks for the bug report. Can you provide a small script that reproduces the error you're seeing? It would be great to know what version of AI2-THOR you're using.

[cghAndy]

My version of AI2-THOR is 5.0.0, and using dataset procthor.

a small script from my code:

def __init__(self, args=None):
       self.args = args
       self.dataset = self.load_dataset()[args.mode]
       self.controller = Controller(scene=self.dataset[0], renderDepthImage=self.args.is_depth,renderInstanceSegmentation=True)
       if self.args.collect_data:
           self.collect_data()

def load_dataset(self) -> prior.DatasetDict:
       data = {}
       dic = {"train": 10000, "val": 1000, "test": 1000}
       for split, size in [("train", 10_000), ("val", 1_000), ("test", 1_000)]:
           with gzip.open(f"dataset/{split}.jsonl.gz", "r") as f:
               houses = [line for line in tqdm(f, total=size, desc=f"Loading {split}")]
           data[split] = LazyJsonDataset(data=houses, dataset="procthor-dataset", split=split)
       return prior.DatasetDict(**data)

def teleport(self, position, rotation, horizon):
       event = self.controller.step(action="Teleport", position=position, rotation=rotation, horizon=horizon)
       return _event_

def get_bounding_box(self, event, objectId):
       return event.instance_detections2D.[objectId]

def get_reachable_position(self):
       event = self.controller.step(action="GetReachablePositions")
       return event.metadata["actionReturn"]

def collect_data(self):
       self.house = self.dataset[idx]
       self.controller.reset(scene=self.house, renderDepthImage=self.args.is_depth, renderInstanceSegmentation=True)
       reachable_position = self.get_reachable_position()
       for position in reachable_position:
                   rotation = random.choice([{'x': 0, 'y': 0, 'z': 0}, {'x': 0, 'y': 90, 'z': 0}, {'x': 0, 'y': 180, 'z': 0}, {'x': 0, 'y': 270, 'z': 0}])
                   horizon = random.choice([0, 30, -30])
                   event = self.teleport(position, rotation, horizon)
                   for obj in event.metadata["objects"]:
                       if obj['visible']:
                              bbox = self.get_bounding_box(event, obj["objectId"])

[sarthakchittawar]

You can use controller.events.class_detections2D to get the 2D bounding boxes instead. Works for me.