From eee76ca0bebe5ffdeee1a36df5811e1befe0eb45 Mon Sep 17 00:00:00 2001 From: Maksymilian Wojnar Date: Mon, 26 Feb 2024 10:08:46 +0100 Subject: [PATCH] Update readme with a multiple agents section --- README.md | 3 +-- docs/source/getting_started.rst | 35 +++++++++++++++++++++++++++++++++ docs/source/index.rst | 4 +--- 3 files changed, 37 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 7e8c504..3841900 100644 --- a/README.md +++ b/README.md @@ -73,8 +73,7 @@ Our library is built on top of JAX, a high-performance numerical computing libra RL algorithms efficiently. It provides powerful transformations, including JIT compilation, automatic differentiation, vectorization, and parallelization. Our library is fully compatible with DeepMind's JAX ecosystem, granting access to state-of-the-art RL models and helper libraries. JIT compilation significantly accelerates execution and ensures -portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications. - +portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications. JAX offers another benefit through its robust pseudorandom number generator system, employed in our library to guarantee result reproducibility. This critical aspect of scientific research is frequently underestimated but remains highly significant. diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst index 46ec54c..7bacf8a 100644 --- a/docs/source/getting_started.rst +++ b/docs/source/getting_started.rst @@ -117,6 +117,41 @@ flag to ``False`` in the ``sample`` method: action = rl.sample(*env_state, is_training=False) +Interaction with multiple agents +-------------------------------- + +Reinforced-lib allows you to use multiple agent instances in the same environment. This feature is useful when you want +to train multiple agents in parallel or use multiple agents to solve the same problem. To achieve this, you need to +initialize the instances of the agents by calling the ``init`` method of the ``RLib`` class a certain number of times: + +.. code-block:: python + + rl = RLib(..) + + for _ in range(n_agents): + rl.init() + +Reproducibility +~~~~~~~~~~~~~~~ + +JAX is focused on reproducibility, and it provides a robust pseudo-random number generator (PRNG) that allows you to +control the randomness of the computations. PRNG requires setting the random seed to ensure that the results of the +computation are reproducible. Reinforced-lib provides an API for setting the random seed for the JAX library. +You can set the seed by providing the ``seed`` parameter when creating the instance of the agent: + +.. code-block:: python + + rl = RLib(...) + rl.init(seed=123) + +The seed is initially configured as 42 and the ``init`` method is triggered automatically after the first sampling call. +It eliminates the need to manually call the ``init`` method unless you want to provide custom seed, thus ensuring +reproducibility. + +.. note:: + + Remember that the reproducibility of the computations is guaranteed only for the agents from Reinforced-lib. + You have to ensure that the environment you use is reproducible as well. Loggers ------- diff --git a/docs/source/index.rst b/docs/source/index.rst index 3eff3be..7fe65cb 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -40,9 +40,7 @@ The figure below provides a visual representation of Reinforced-lib and the data JAX Backend ----------- -Our library is built on top of JAX, a high-performance numerical computing library. JAX makes it easy to implement RL algorithms efficiently. It provides powerful transformations, including JIT compilation, automatic differentiation, vectorization, and parallelization. Our library is fully compatible with DeepMind's JAX ecosystem, granting access to state-of-the-art RL models and helper libraries. JIT compilation significantly accelerates execution and ensures portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications. - -JAX offers another benefit through its robust pseudorandom number generator system, employed in our library to guarantee result reproducibility. This critical aspect of scientific research is frequently underestimated but remains highly significant. +Our library is built on top of JAX, a high-performance numerical computing library. JAX makes it easy to implement RL algorithms efficiently. It provides powerful transformations, including JIT compilation, automatic differentiation, vectorization, and parallelization. Our library is fully compatible with DeepMind's JAX ecosystem, granting access to state-of-the-art RL models and helper libraries. JIT compilation significantly accelerates execution and ensures portability across different architectures (CPUs, GPUs, TPUs) without requiring code modifications. JAX offers another benefit through its robust pseudorandom number generator system, employed in our library to guarantee result reproducibility. This critical aspect of scientific research is frequently underestimated but remains highly significant. Edge Device Export ------------------