Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make TopDownMultiChannel support ScenarioEnv #498

Merged
merged 8 commits into from
Oct 18, 2023
Merged

Make TopDownMultiChannel support ScenarioEnv #498

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
739fd07
works!
pimpale Sep 1, 2023
3eac8b2
Merge branch 'main' of https://github.com/metadriverse/metadrive
pimpale Sep 18, 2023
44ef311
add preliminary support for top down view when using a scenario
pimpale Sep 19, 2023
94e1482
format
pimpale Sep 19, 2023
eda5586
remove extraeous change
pimpale Sep 19, 2023
2f141be
draw trajectories
pimpale Sep 19, 2023
7854fac
implement fixes, merge main
pimpale Sep 22, 2023
75a8f4d
format and fix bug
pimpale Sep 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion metadrive/component/road_network/edge_road_network.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from metadrive.utils.math import get_boxes_bounding_box
from metadrive.utils.pg.utils import get_lanes_bounding_box

lane_info = namedtuple("neighbor_lanes", "lane entry_lanes exit_lanes left_lanes right_lanes")
lane_info = namedtuple("edge_lane", ["lane", "entry_lanes", "exit_lanes", "left_lanes", "right_lanes"])


class EdgeRoadNetwork(BaseRoadNetwork):
Expand Down
5 changes: 5 additions & 0 deletions metadrive/manager/scenario_traffic_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from metadrive.component.static_object.traffic_object import TrafficCone, TrafficBarrier
from metadrive.component.traffic_participants.cyclist import Cyclist
from metadrive.component.traffic_participants.pedestrian import Pedestrian
from metadrive.component.vehicle.base_vehicle import BaseVehicle
from metadrive.component.vehicle.vehicle_type import get_vehicle_type, reset_vehicle_type_count
from metadrive.constants import DEFAULT_AGENT
from metadrive.manager.base_manager import BaseManager
Expand Down Expand Up @@ -162,6 +163,10 @@ def sdc_object_id(self):
def current_scenario_length(self):
return self.engine.data_manager.current_scenario_length

@property
def vehicles(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there are more objects in a traffic scenario now, drawing only vehicles is not enough. It would be better to draw all objects including cones, barriers, pedestrians, and bikes. So I suggest drawing all objects like what we did in the Top-Down renderer. This can be done by accessing all objects via engine._spawned_objects or engine.get_objects() and draw them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done so now!

return list(self.engine.get_objects(filter=lambda o: isinstance(o, BaseVehicle)).values())

def spawn_vehicle(self, v_id, track):
state = parse_object_state(track, self.episode_step)

Expand Down
55 changes: 44 additions & 11 deletions metadrive/obs/top_down_obs_multi_channel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,23 @@
import numpy as np

from metadrive.component.vehicle.base_vehicle import BaseVehicle
from metadrive.scenario.scenario_description import ScenarioDescription
from metadrive.utils.interpolating_line import InterpolatingLine
from metadrive.component.lane.point_lane import PointLane
from metadrive.constants import Decoration, DEFAULT_AGENT
from metadrive.obs.top_down_obs import TopDownObservation
from metadrive.obs.top_down_obs_impl import WorldSurface, COLOR_BLACK, VehicleGraphics, LaneGraphics, \
ObservationWindowMultiChannel
from metadrive.utils import import_pygame, clip

from metadrive.component.road_network.node_road_network import NodeRoadNetwork
from metadrive.component.vehicle_navigation_module.node_network_navigation import NodeNetworkNavigation
from metadrive.component.vehicle_navigation_module.edge_network_navigation import EdgeNetworkNavigation
from metadrive.component.vehicle_navigation_module.trajectory_navigation import TrajectoryNavigation

pygame = import_pygame()
COLOR_WHITE = pygame.Color("white")
DEFAULT_TRAJECTORY_LANE_WIDTH = 5


class TopDownMultiChannel(TopDownObservation):
Expand Down Expand Up @@ -106,16 +115,30 @@ def draw_map(self) -> pygame.Surface:
self.canvas_background.move_display_window_to(centering_pos)
self.canvas_road_network.move_display_window_to(centering_pos)

# self.draw_navigation(self.canvas_navigation)
self.draw_navigation(self.canvas_background, (64, 64, 64))
if isinstance(self.target_vehicle.navigation, NodeNetworkNavigation):
self.draw_navigation_node(self.canvas_background, (64, 64, 64))
elif isinstance(self.target_vehicle.navigation, EdgeNetworkNavigation):
# TODO: draw edge network navigation
pass
elif isinstance(self.target_vehicle.navigation, TrajectoryNavigation):
self.draw_navigation_trajectory(self.canvas_background, (64, 64, 64))

if isinstance(self.road_network, NodeRoadNetwork):
for _from in self.road_network.graph.keys():
decoration = True if _from == Decoration.start else False
for _to in self.road_network.graph[_from].keys():
for l in self.road_network.graph[_from][_to]:
two_side = True if l is self.road_network.graph[_from][_to][-1] or decoration else False
LaneGraphics.LANE_LINE_WIDTH = 0.5
LaneGraphics.display(l, self.canvas_background, two_side)
elif hasattr(self.engine, "map_manager"):
for data in self.engine.map_manager.current_map.blocks[-1].map_data.values():
if ScenarioDescription.POLYLINE in data:
LaneGraphics.display_scenario(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember what the figure drawn with this code looks like, so I am not sure if display_scenario achieves what you want. If not, just check this implementation in the if semantic_map:

metadrive/metadrive/obs/top_down_renderer.py

Line 52 in cc26984

if semantic_map:

It gives you filled polygons to show maps and lines to represent lines like this:

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I use LaneGraphics.display_scenario_line to draw the lines, I get this image:

image

Ideally, it would have the same spacing between stripes as LaneGraphics.striped_line, so I'll work on ensuring that.

InterpolatingLine(data[ScenarioDescription.POLYLINE]), data.get("type", None),
self.canvas_background
)

for _from in self.road_network.graph.keys():
decoration = True if _from == Decoration.start else False
for _to in self.road_network.graph[_from].keys():
for l in self.road_network.graph[_from][_to]:
two_side = True if l is self.road_network.graph[_from][_to][-1] or decoration else False
LaneGraphics.LANE_LINE_WIDTH = 0.5
LaneGraphics.display(l, self.canvas_background, two_side)
self.canvas_road_network.blit(self.canvas_background, (0, 0))
self.obs_window.reset(self.canvas_runtime)
self._should_draw_map = False
Expand All @@ -142,7 +165,13 @@ def draw_scene(self):
ego_heading = vehicle.heading_theta
ego_heading = ego_heading if abs(ego_heading) > 2 * np.pi / 180 else 0

for v in self.engine.traffic_manager.vehicles:
vehicles = []
if hasattr(self.engine, "traffic_manager"):
vehicles = self.engine.traffic_manager.vehicles
elif hasattr(self.engine, "scenario_traffic_manager"):
vehicles = self.engine.scenario_traffic_manager.vehicles
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said before, let's draw all other things besides vehicles. A suggestion here is that we can actually cancel this if condition. For both PGEnvironment and ScenarioEnvironment, all objects are actually in the engine._spawned_objects. What a manager does is a set of rules to add, delete, or actuate objects. Thus we can access all objects through env.engine directly, regardless of what the traffic manager is.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!


for v in vehicles:
if v is vehicle:
continue
h = v.heading_theta
Expand Down Expand Up @@ -256,13 +285,17 @@ def observe(self, vehicle: BaseVehicle):
img = np.clip(img, 0, 255)
return np.transpose(img, (1, 0, 2))

def draw_navigation(self, canvas, color=(128, 128, 128)):
def draw_navigation_node(self, canvas, color=(128, 128, 128)):
checkpoints = self.target_vehicle.navigation.checkpoints
for i, c in enumerate(checkpoints[:-1]):
lanes = self.road_network.graph[c][checkpoints[i + 1]]
for lane in lanes:
LaneGraphics.draw_drivable_area(lane, canvas, color=color)

def draw_navigation_trajectory(self, canvas, color=(128, 128, 128)):
lane = PointLane(self.target_vehicle.navigation.checkpoints, DEFAULT_TRAJECTORY_LANE_WIDTH)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already an ego car trajectory in the simulator called engine.map_manager.current_sdc_route. But it is ok to use either of the two.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense!

LaneGraphics.draw_drivable_area(lane, canvas, color=color)

def _get_stack_indices(self, length, frame_skip=None):
frame_skip = frame_skip or self.frame_skip
num = int(math.ceil(length / frame_skip))
Expand Down
Loading