Flow

Flow is a computational framework for deep RL and control experiments for traffic microsimulation.

See our website for more information on the application of Flow to several mixed-autonomy traffic scenarios. Other results and videos are available as well.

Installations

• Documentation
• Installation instructions
• Tutorials
• Binder Build (beta)

Experiments

• Installation:

  • SUMO 1.6.0 Linux ubuntu (please use the exact version for reproducibility)
    • SUMO 1.6.0 needs to be installed by building from source
    • Download source code from here: https://sourceforge.net/projects/sumo/files/sumo/
    • Sumo-src-1.6.0.zip or sumo-src-1.6.0.tar.gz
    • Check the instructions here https://sumo.dlr.de/docs/Installing/Linux_Build.html
  • Flow

    conda env create -f environment.yml
    conda activate flow
    python setup.py develop

• Training:

  • You can train using the file flow/benchmarks/rllib/ppo_runner.py and passing in a benchmark name. I recommend passing in merge4_Sim as a first step. Note that this will take several hours to train.
  • For Centralized experiments, you should run the command from the flow/benchmarks directory. The specific command you should run is:

    python rllib/ppo_runner.py --benchmark_name merge4_Sim --num_cpus  number_of_cores --lr 5e-5 --num_rollouts 2

  • For Distributed experiments, please first go to the directory and then run the corresponding scripts:

    cd examples/rllib/multiagent_exps
    python multiagent_merge4_Merge4_Collaborate_lrschedule.py 

• Evaluation

  • Tensorboard You can view the results by running

    tensorboard --logdir results_dir

    • The results should be stored in your ~/ray_results directory after training (e.g. ~/ray_results/merge_4/PPO/exp1) You can view the results by opening a browser to localhost:6006
    • Install tensorboard in base and run this command in base.
  • Policy Visualizer
    • Once it's finished try visualizing it with the file flow/visualize/new_rllib_visualizer.py. You do so by passing in a checkpoint directory and a checkpoint number. You should run the command from the MITC directory. The specific command you should run is:

    python flow/visualize/new_rllib_visualizer.py checkpoint_dir checkpoint_num

• Experiments Here we will give an overview of the necessary python classes related to the experiment in our AAMAS paper “Scalable Multiagent Driving Policies For Reducing Traffic Congestion”

Architecture	Section in AAMAS	Description	Benchmark/File name
Decentrailzed	Simple Merge	Full State Augmentation	multiagent_merge4_Merge4_Collaborate_lrschedule.py
Centralized	Simple Merge	Human	flow.benchmarks.merge4_IDM
Centralized	Simple Merge	Flow Reward	flow.benchmarks.merge4_Sim
Centralized	Simple Merge	Outflow Reward	flow.benchmarks.merge4_Sim_Arrive.py
Centralized	Simple Merge	Average Speed Reward	flow.benchmarks.merge4_Sim_AvgVel
Centralized	I-696	Window, Ouflow	flow.benchmarks.1merge_Window_transfer_Arrive
Centralized	I-696	Window, AvgVel	flow.benchmarks.1merge_Window_transfer_AvgVel
Centralized	I-696	Entire, Outflow	flow.benchmarks.1merge_horizon2000_warmup0_simstep05_flow2000_merge200_dePart10_Arrive
Centralized	I-696	Entire, AvgVel	flow.benchmarks.1merge_horizon2000_warmup0_simstep05_flow2000_merge200_dePart10_AvgVel
Centralized	I-696	ZeroShot	load policies trained in simple merge and test them in I-696

Technical questions

If you have a bug, please report it. Otherwise, join the Flow Users group on Slack! You'll recieve an email shortly after filling out the form.

Getting involved

We welcome your contributions.

• Please report bugs and improvements by submitting GitHub issue.
• Submit your contributions using pull requests. Please use this template for your pull requests.

Citing Flow

If you are interested in extending our research, please consider citing our paper:

@article{cui2021scalable,
  title={Scalable multiagent driving policies for reducing traffic congestion},
  author={Cui, Jiaxun and Macke, William and Yedidsion, Harel and Goyal, Aastha and Urielli, Daniel and Stone, Peter},
  journal={arXiv preprint arXiv:2103.00058},
  year={2021}
}

If you use Flow for academic research, you are highly encouraged to cite the Flow paper:

C. Wu, A. Kreidieh, K. Parvate, E. Vinitsky, A. Bayen, "Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control," CoRR, vol. abs/1710.05465, 2017. [Online]. Available: https://arxiv.org/abs/1710.05465

If you use the benchmarks, you are highly encouraged to cite our paper:

Vinitsky, E., Kreidieh, A., Le Flem, L., Kheterpal, N., Jang, K., Wu, F., ... & Bayen, A. M, Benchmarks for reinforcement learning in mixed-autonomy traffic. In Conference on Robot Learning (pp. 399-409). Available: http://proceedings.mlr.press/v87/vinitsky18a.html

Contributors

Flow is supported by the Mobile Sensing Lab at UC Berkeley and Amazon AWS Machine Learning research grants. The contributors are listed in Flow Team Page.