Update readme and documentation

README.md

@@ -14,11 +14,11 @@
 [rtd-badge]: https://readthedocs.org/projects/reinforced-lib/badge/?version=latest
 [documentation]: https://reinforced-lib.readthedocs.io/en/latest/
 
-**Introducing Reinforced-lib:** a lightweight Python library for the rapid development of RL solutions. It is open-source, 
-prioritizes ease of use, provides comprehensive documentation, and offers both deep reinforcement learning 
-(DRL) and classic non-neural agents. Built on [JAX](https://jax.readthedocs.io/en/latest/), it facilitates exporting 
-trained models to embedded devices, and makes it great for research and prototyping with RL algorithms. Access to JAX's 
-just-in-time (JIT) compilation ensures high-performance results.
+**Introducing Reinforced-lib:** a lightweight Python library for the rapid development of reinforcement-learning (RL) 
+solutions. It is open-source, prioritizes ease of use, provides comprehensive documentation, and offers both deep 
+reinforcement learning (DRL) and classic non-neural agents. Built on [JAX](https://jax.readthedocs.io/en/latest/), 
+it facilitates exporting trained models to embedded devices, and makes it great for research and prototyping with RL 
+algorithms. Access to JAX's just-in-time (JIT) compilation ensures high-performance results.
 
 ## Installation
 
@@ -41,7 +41,7 @@ In the spirit of making Reinforced-lib a lightweight solution, we include only t
 requirements. To fully benefit from Reinforced-lib's conveniences, such as TF Lite export, install with the "full" suffix:
 
 ```bash
-pip3 install ".[full]"
+pip install ".[full]"
 ```
 
 ## Key components
@@ -50,8 +50,8 @@ Reinforced-lib facilitates seamless interaction between RL agents and the enviro
 within of the library, represented in the API as different modules.
 
 - **RLib** - The core module which provides a simple and intuitive interface to manage agents, use extensions, 
-  and configure the logging system. Even if you're not a reinforcement learning (RL) expert, *RLib* makes it easy to 
-  implement the agent-environment interaction loop.
+  and configure the logging system. Even if you're not an RL expert, *RLib* makes it easy to  implement the 
+  agent-environment interaction loop.
 
 - **Agents** - Choose from a variety of RL agents available in the *Agents* module. These agents are designed to be 
   versatile and work with any environment. If needed, you can even create your own agents using our documented recipes.
@@ -70,11 +70,15 @@ The figure below provides a visual representation of Reinforced-lib and the data
 ## JAX Backend
 
 Our library is built on top of JAX, a high-performance numerical computing library. JAX makes it easy to implement 
-RL algorithms efficiently. It provides powerful transformations, including JIT compilation,  automatic differentiation, 
+RL algorithms efficiently. It provides powerful transformations, including JIT compilation, automatic differentiation, 
 vectorization, and parallelization. Our library is fully compatible with DeepMind's JAX ecosystem, granting access to 
 state-of-the-art RL models and helper libraries. JIT compilation significantly accelerates execution and ensures 
 portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications.
 
+JAX offers another benefit through its robust pseudorandom number generator system, employed in our library to 
+guarantee result reproducibility. This critical aspect of scientific research is frequently underestimated but 
+remains highly significant.
+
 ## Edge Device Export
 
 Reinforced-lib is designed to work seamlessly on wireless, low-powered devices, where resources are limited. It's the 

docs/source/conf.py

@@ -20,7 +20,7 @@
 # -- Project information -----------------------------------------------------
 
 project = 'reinforced-lib'
-copyright = '2023, Maksymilian Wojnar, Wojciech Ciężobka'
+copyright = '2024, Maksymilian Wojnar, Wojciech Ciężobka'
 author = 'Maksymilian Wojnar, Wojciech Ciężobka'
 
 

docs/source/custom_agents.rst

@@ -371,7 +371,7 @@ There are also other utility functions that can make it easier to implement DRL
 ``init`` and ``forward`` methods which are used to initialize the network and to perform a forward pass through the
 network. You can find more information about these functions in the :ref:`documentation <Utils>`.
 
-Our Python library also includes a pre-built :ref:`experience replay buffer <Experience Replay>`, which is commonly
+Our Python library also includes a pre-built :ref:`experience replay buffer <Experience replay>`, which is commonly
 utilized in DRL agents. The following code provides an illustrative example of how to use this utility:
 
 .. code-block:: python

docs/source/examples.rst

@@ -151,7 +151,7 @@ The complete, runnable code can be copy pasted from the following snippet:
 Other examples
 ==============
 
-We provide a few more examples of Reinforced-lib and Gymnasium integration in the ``examples`` directory of the Reinforced-lib repository. The examples include the training of the DQN agent in the Cart Pole environment (described above) and the training of the DDPG agent in the Pendulum environment. The examples are fully runnable and can be used as a starting point for your own reinforcement learning experiments with Reinforced-lib and Gymnasium.
+We provide a few more examples of Reinforced-lib and Gymnasium integration in the `examples <https://github.com/m-wojnar/reinforced-lib/tree/main/examples>`_ directory of the Reinforced-lib repository. The examples include the training of the DQN agent in the `Cart Pole environment <https://github.com/m-wojnar/reinforced-lib/tree/main/examples/cart-pole>`_ (described above) and the training of the DDPG agent in the `Pendulum environment <https://github.com/m-wojnar/reinforced-lib/tree/main/examples/pendulum>`_. The examples are fully runnable and can be used as a starting point for your own reinforcement learning experiments with Reinforced-lib and Gymnasium.
 
 
 .. _ns3_connection:
@@ -470,4 +470,4 @@ You can try running the following commands to test the Reinforced-lib rate adapt
 Source code of the example
 ===========================
 
-The complete, runnable code can be found in the ``examples/ns-3-ra`` directory of the Reinforced-lib repository. The example provides many useful scripts for reproducing our experiments and can be used as a starting point for your own reinforcement learning experiments with Reinforced-lib and ns-3. We also encourage you to see another example - implementation of the `centralized contention window optimization with DRL (CCOD) <https://ieeexplore.ieee.org/document/9417575?denied=>`_ in the ``examples/ns-3-ccod`` directory which presents a deep reinforcement learning scenario with Reinforced-lib and ns-3.
+The complete, runnable code can be found in the `examples/ns-3-ra <https://github.com/m-wojnar/reinforced-lib/tree/main/examples/ns-3-ra>`_ directory of the Reinforced-lib repository. The example provides many useful scripts for reproducing our experiments and can be used as a starting point for your own reinforcement learning experiments with Reinforced-lib and ns-3. We also encourage you to see another example - implementation of the `centralized contention window optimization with DRL (CCOD) <https://ieeexplore.ieee.org/document/9417575?denied=>`_ in the ``examples/ns-3-ccod`` directory which presents a deep reinforcement learning scenario with Reinforced-lib and ns-3.

docs/source/extensions.rst

@@ -16,12 +16,29 @@ BaseExt
     :members:
 
 
+BasicMab
+--------
+
+.. currentmodule:: reinforced_lib.exts.basic_mab
+
+.. autoclass:: BasicMab
+    :show-inheritance:
+    :members:
+
+
 Gymnasium
 ---------
 
-
 .. currentmodule:: reinforced_lib.exts.gymnasium
 
 .. autoclass:: Gymnasium
     :show-inheritance:
     :members:
+
+
+Extension utils
+---------------
+
+.. automodule:: reinforced_lib.exts.utils
+    :members:
+    :no-index:

docs/source/index.rst

@@ -15,14 +15,14 @@ Welcome to Reinforced-lib's documentation!
 .. image:: https://readthedocs.org/projects/reinforced-lib/badge/?version=latest
     :target: https://reinforced-lib.readthedocs.io/
 
-**Introducing Reinforced-lib:** a lightweight Python library for rapid development of RL solutions. It is open-source, prioritizes ease of use, provides comprehensive documentation, and offers both deep reinforcement learning (DRL) and classic non-neural agents. Built on `JAX <https://jax.readthedocs.io/en/latest/>`_, it facilitates exporting trained models to embedded devices, and makes it great for research and prototyping with RL algorithms. Access to JAX's JIT functionality ensure high-performance results.
+**Introducing Reinforced-lib:** a lightweight Python library for rapid development of reinforcement-learning (RL) solutions. It is open-source, prioritizes ease of use, provides comprehensive documentation, and offers both deep reinforcement learning (DRL) and classic non-neural agents. Built on `JAX <https://jax.readthedocs.io/en/latest/>`_, it facilitates exporting trained models to embedded devices, and makes it great for research and prototyping with RL algorithms. Access to JAX's JIT functionality ensure high-performance results.
 
 Key components
 --------------
 
 Reinforced-lib facilitates seamless interaction between RL agents and the environment. Here are the key components within of the library, represented in the API as different modules.
 
-* **RLib** -- The core module which provides a simple and intuitive interface to manage agents, use extensions, and configure the logging system. Even if you're not a reinforcement learning (RL) expert, *RLib* makes it easy to implement the agent-environment interaction loop.
+* **RLib** -- The core module which provides a simple and intuitive interface to manage agents, use extensions, and configure the logging system. Even if you're not an RL expert, *RLib* makes it easy to implement the agent-environment interaction loop.
 
 * **Agents** -- Choose from a variety of RL agents available in the *Agents* module. These agents are designed to be versatile and work with any environment. If needed, you can even create your own agents using our documented recipes.
 
@@ -42,6 +42,8 @@ JAX Backend
 
 Our library is built on top of JAX, a high-performance numerical computing library. JAX makes it easy to implement RL algorithms efficiently. It provides powerful transformations, including JIT compilation, automatic differentiation, vectorization, and parallelization. Our library is fully compatible with DeepMind's JAX ecosystem, granting access to state-of-the-art RL models and helper libraries. JIT compilation significantly accelerates execution and ensures portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications.
 
+JAX offers another benefit through its robust pseudorandom number generator system, employed in our library to guarantee result reproducibility. This critical aspect of scientific research is frequently underestimated but remains highly significant.
+
 Edge Device Export
 ------------------
 

docs/source/loggers.rst

@@ -7,21 +7,21 @@ This module is a set of loggers. You can either choose one of our built-in logge
 your logger with the help of the :ref:`Custom loggers` guide.
 
 
-BaseLogger
-----------
+LogsObserver
+------------
 
-.. currentmodule:: reinforced_lib.logs.base_logger
+.. currentmodule:: reinforced_lib.logs.logs_observer
 
-.. autoclass:: BaseLogger
+.. autoclass:: LogsObserver
     :members:
 
 
-LogsObserver
-------------
+BaseLogger
+----------
 
-.. currentmodule:: reinforced_lib.logs.logs_observer
+.. currentmodule:: reinforced_lib.logs.base_logger
 
-.. autoclass:: LogsObserver
+.. autoclass:: BaseLogger
     :members:
 
 

docs/source/utils.rst

@@ -11,14 +11,14 @@ JAX
 .. automodule:: reinforced_lib.utils.jax_utils
     :members:
 
-Experience Replay
+Experience replay
 -----------------
 
 .. automodule:: reinforced_lib.utils.experience_replay
     :members:
 
-Particle filter (Core)
-----------------------
+Particle filter
+---------------
 
 .. automodule:: reinforced_lib.utils.particle_filter
     :show-inheritance: