Reinforcement-Learning-course

Advanced course on Reinforcement Learning.

Sylabus

Module 1: Introduction to Reinforcement Learning

1. Overview of Reinforcement Learning and its applications
2. Markov Decision Processes (MDPs) and Bellman Equations
3. Q-Learning and SARSA algorithms

Module 2: Temporal-Difference methods

1. TD Learning
2. TD prediction
3. SARSA and TD control

Module 3: Monte Carlo Methods

1. First-Visit Monte Carlo and Every-Visit Monte Carlo methods
2. On-Policy and Off-Policy methods
3. Importance Sampling

Module 4: Function Approximation

1. Introduction to function approximation for Reinforcement Learning
2. Overview of Deep Reinforcement Learning
3. Hands-on experience with Gymnasium environment

Module 5: Project Work and Conclusion

1. Final project: students will work on a real-world Reinforcement Learning problem using the techniques and tools learned in the course
2. Course conclusion and future directions in Reinforcement Learning research

References:

1. "Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto (2018)
2. Python implementation based on the book "Reinforcement Learning: An Introduction" by Richard S. Sutton and Andrew G. Barto (2018)
3. "Deep Reinforcement Learning Hands-On" by Maxim Lapan (2018)
4. Code for the book "Deep Reinforcement Learning Hands-On" by Maxim Lapan (2018)
5. Gymnasium environment (https://gymnasium.farama.org/)
6. TensorFlow documentation (https://www.tensorflow.org/guide)
7. PyTorch documentation (https://pytorch.org/docs/stable/index.html)
8. AlphaGo Documentary
9. Monte Carlo Tree Search Another Introduction
10. Stable Baselines
11. Application to cartpole. Use of vectorized environments.
12. AlphaGO to Muzero
13. Trackmania bot Training an AI to learn to win at Trackmania game.
14. Code to train a Trackmania bot

Multi-Agent Reinforcement Learning (MARL)

1. PettingZoo Multi-agent reinforcement learning environment, compatible with Gymnasium.
2. Tianshou A PyTorch library for deep reinforcement learning.
3. TorchRL A PyTorch library for reinforcement learning.
4. BenchMARL BenchMARL is a Multi-Agent Reinforcement Learning (MARL) training library created to enable reproducibility and benchmarking across different MARL algorithms and environments.
5. VMAS - Vectorized Multi-Agent Reinforcement Learning Simulator VMAS is a simulator for multi-agent reinforcement learning (MARL) that supports vectorized environments and is designed to be fast and easy to use.