A Soft Actor Policy based model free off-policy network to control the steering and throttle of car while drifting at high speeds.
-
Our model will be trained using the Soft-Actor Critic (SAC), which optimizes the error loss (anticipated return - prediction) and maximizes entropy using an off-policy learning strategy to perform better in continuous domains.
-
Control policy is the actor in SAC, while value and Q-network will function as critics.
-
The basic goal of the actor is to maximize reward while minimizing entropy (measure of randomness in the policy - more exploration)
- Ubuntu 20.04
- Conda : Package and environment manager
- Python 3.8
- Pytorch
- Pygame
We are using CARLA 0.9.5 as our version for our simulation.
Please download the the simulator from this drive
Extract the folder in your Downloads directory.
If you have a dual GPU setup , please enter the following command to enable your secondary graphics card as the primary one.
export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"
Add Path to your bash file
export PYTHONPATH=$PYTHONPATH:~/Downloads/CARLA_DRIFT_0.9.5/PythonAPI/carla/dist/carla-0.9.5-py3.5-linux-x86_64.egg
export PYTHONPATH=$PYTHONPATH:~/Downloads/CARLA_DRIFT_0.9.5/PythonAPI/carla/
To open and run the simulator please enter the below commands open a new terminal
cd Downloads/CARLA_DRIFT_0.9.5
./CarlaUE4.sh /Game/Carla/ExportedMaps/simple
This will show up the map.If you want to spawn vehicles and manually control the vehicle in the above map please enter the below commands.
Open a New terminal
cd Downloads/CARLA_DRIFT_0.9.5/PythonAPI/examples
./spawn_npc.py
This will spawn vehicels in the map.
To control a vehicle in the environment, enter the below commands.
Open a New terminal
cd Downloads/CARLA_DRIFT_0.9.5/PythonAPI/examples
./manual_control.py
#TODO
Need to take reference trajectories data for the above map and train with the SAC.