Merge pull request #618 from BDonnot/issue_616

Issue #616

CHANGELOG.rst

@@ -40,6 +40,8 @@ Work kind of in progress
 
 Next release
 ---------------------------------
+- TODO bug on maintenance starting at midnight (they are not correctly handled in the observation)
+  => cf script test_issue_616
 - TODO Notebook for tf_agents
 - TODO Notebook for acme
 - TODO Notebook using "keras rl" (see https://keras.io/examples/rl/ppo_cartpole/)
@@ -48,7 +50,8 @@ Next release
 - TODO jax everything that can be: create a simple env based on jax for topology manipulation, without
   redispatching or rules
 - TODO backend in jax, maybe ?
-
+- TODO done and truncated properly handled in gym_compat module (when game over
+  before the end it's probably truncated and not done) 
 
 [1.10.3] - 2024-xx-yy
 -------------------------
@@ -65,10 +68,23 @@ Next release
   (because  it should always have been like this)
 - [FIXED] a bug in the `MultiFolder` and `MultifolderWithCache` leading to the wrong 
   computation of `max_iter` on some corner cases
+- [FIXED] issue on `seed` and `MultifolderWithCache` which caused 
+  https://github.com/rte-france/Grid2Op/issues/616 
+- [FIXED] another issue with the seeding of `MultifolderWithCache`: the seed was not used
+  correctly on the cache data when calling `chronics_handler.reset` multiple times without 
+  any changes
 - [ADDED] possibility to skip some step when calling `env.reset(..., options={"init ts": ...})`
 - [ADDED] possibility to limit the duration of an episode with `env.reset(..., options={"max step": ...})`
 - [ADDED] possibility to specify the "reset_options" used in `env.reset` when
   using the runner with `runner.run(..., reset_options=xxx)`
+- [ADDED] the time series now are able to regenerate their "random" part 
+  even when "cached" thanks to the addition of the `regenerate_with_new_seed` of the 
+  `GridValue` class (in public API)
+- [ADDED] `MultifolderWithCache` now supports `FromHandlers` time series generator
+- [IMPROVED] the documentation on the `time series` folder.
+- [IMPROVED] now the "maintenance from json" (*eg* the `JSONMaintenanceHandler` or the 
+  `GridStateFromFileWithForecastsWithMaintenance`) can be customized with the day 
+  of the week where the maintenance happens (key `maintenance_day_of_week`)
 
 [1.10.2] - 2024-05-27
 -------------------------

docs/chronics.rst

@@ -54,26 +54,158 @@ come from the :class:`grid2op.GridValue` and are detailed in the
 :func:`GridValue.forecasts` method.
 
 
-More control on the chronics
+More control on the time series
 -------------------------------
 We explained, in the description of the :class:`grid2op.Environment` in sections
 :ref:`environment-module-chronics-info` and following how to have more control on which chronics is used,
 with steps are used within a chronics etc. We will not detailed here again, please refer to this page
 for more information.
 
-However, know that you can have a very detailed control on which chronics are used:
+However, know that you can have a very detailed control on which time series using the `options`
+kwargs of a call to `env.reset()` (or the `reset_otions` kwargs when calling the 
+`runner.run()`) : 
 
-- use `env.set_id(THE_CHRONIC_ID)` (see :func:`grid2op.Environment.Environment.set_id`) to set the id of the
-  chronics you want to use
-- use `env.chronics_handler.set_filter(a_function)` (see :func:`grid2op.Chronics.GridValue.set_filter`)
+
+Use a specific time serie for an episode
+*******************************************
+
+To use a specific time series for a given episode, you can use 
+`env.reset(options={"time serie id": THE_ID_YOU_WANT)`.
+
+For example:
+
+.. code-block:: python
+
+  import grid2op
+  env_name = "l2rpn_case14_sandbox"
+  env = grid2op.make(env_name)
+
+  # you can use an int:
+  obs = env.reset(options={"time serie id": 0})
+
+  # or the name of the folder (for most grid2op environment)
+  obs = env.reset(options={"time serie id": "0000"})  # for l2rpn_case14_sandbox
+
+  # for say l2rpn_neurips_2020_track1
+  # obs = env.reset(options={"time serie id": "Scenario_august_008"})
+
+  # for say l2rpn_idf_2023
+  # obs = env.reset(options={"time serie id": "2035-04-23_7"})
+
+
+.. note::
+  For oldest grid2op versions (please upgrade if that's the case) you needed to use:
+  `env.set_id(THE_CHRONIC_ID)` (see :func:`grid2op.Environment.Environment.set_id`) to set the id of the
+  chronics you want to use.
+
+
+Skipping the initial few steps
+*******************************
+
+Often the time series provided for an environment always start at the same date and time on 
+the same hour of the day and day of the week. It might not be ideal to learn controler
+with such data or might "burn up" computation time during evaluation.
+
+To do that, you can use the `"init ts"` reset options, for example with:
+
+.. code-block:: python
+
+  import grid2op
+  env_name = "l2rpn_case14_sandbox"
+  env = grid2op.make(env_name)
+
+  # you can use an int:
+  obs = env.reset(options={"init ts": 12})
+
+  # obs will skip the first hour of the time series
+  # 12 steps is equivalent to 1h (5 mins per step in general)
+
+
+.. note::
+
+  For oldest grid2op versions (please upgrade if that's the case) you needed to use:
+  `env.fast_forward_chronics(nb_time_steps)`
+  (see :func:`grid2op.Environment.BaseEnv.fast_forward_chronics`) to skip initial 
+  few steps
+  of a given chronics.
+
+  Please be aware that this "legacy" behaviour has some issues and is "less clear"
+  than the "init ts" above and it can have some weird combination with 
+  `set_max_iter` for example.
+
+
+Limit the maximum length of the current episode
+*************************************************
+
+For most enviroment, the maximum duration of an episode is the equivalent of a week
+(~2020 steps) or a month (~8100 steps) which might be too long for some usecase.
+
+Anyway, if you want to reduce it, you can now do it with the `"max step"` reset
+option like this:
+
+.. code-block:: python
+
+  import grid2op
+  env_name = "l2rpn_case14_sandbox"
+  env = grid2op.make(env_name)
+
+  # you can use an int:
+  obs = env.reset(options={"max step": 2*288})
+
+  # the maximum duration of the episode is now 2*288 steps
+  # the equivalent of two days
+
+.. note::
+
+  For oldest grid2op versions (please upgrade if that's the case) you needed to use:
+  `env.chronics_handler.set_max_iter(nb_max_iter)`
+  (see :func:`grid2op.Chronics.ChronicsHandler.set_max_iter`) to limit the number 
+  of steps within an episode.
+
+  Please be aware that this "legacy" behaviour has some issues and is "less clear"
+  than the "init ts" above and it can have some weird combination with 
+  `fast_forward_chronics` for example.
+
+Discard some time series from the existing folder
+**************************************************
+
+The folder containing the time series for a given grid2op environment often contains
+dozens (thousands sometimes) different time series.
+
+You might want to use only part of them at some point (whether it's some for training and some
+for validation and test, or some for training an agent on a process and some to train the 
+same agent on another process etc.)
+
+Anyway, if you want to do this (on the majority of released environments) you can do it
+thanks to the `env.chronics_handler.set_filter(a_function)`.
+
+For example:
+
+.. code-block:: python
+
+  import re
+  import grid2op
+  env_name = "l2rpn_case14_sandbox"
+  env = grid2op.make(env_name)
+
+  def keep_only_some_ep(chron_name):
+    return re.match(r".*00.*", chron_name) is not None
+
+  env.chronics_handler.set_filter(keep_only_some_ep)
+  li_episode_kept = env.chronics_handler.reset()
+
+
+.. note::
+  For oldest grid2op versions (please upgrade if that's the case) you needed to use:
+  use `env.chronics_handler.set_filter(a_function)` (see :func:`grid2op.Chronics.GridValue.set_filter`)
   to only use certain chronics
+
+
 - use `env.chronics_handler.sample_next_chronics(probas)`
   (see :func:`grid2op.Chronics.GridValue.sample_next_chronics`) to draw at random some chronics
-- use `env.fast_forward_chronics(nb_time_steps)`
-  (see :func:`grid2op.Environment.BaseEnv.fast_forward_chronics`) to skip initial number of steps
-  of a given chronics
-- use `env.chronics_handler.set_max_iter(nb_max_iter)`
-  (see :func:`grid2op.Chronics.ChronicsHandler.set_max_iter`) to limit the number of steps within an episode
+
+Performance gain (throughput)
+********************************
 
 Chosing the right chronics can also lead to some large advantage in terms of computation time. This is
 particularly true if you want to benefit the most from HPC for example. More detailed is given in the

grid2op/Chronics/GSFFWFWM.py

@@ -108,6 +108,14 @@ def initialize(
             self.max_daily_number_per_month_maintenance = dict_[
                 "max_daily_number_per_month_maintenance"
             ]
+
+            if "maintenance_day_of_week" in dict_:
+                self.maintenance_day_of_week = [int(el) for el in dict_[
+                    "maintenance_day_of_week"
+                ]]
+            else:
+                self.maintenance_day_of_week = np.arange(5)
+
         super().initialize(
             order_backend_loads,
             order_backend_prods,
@@ -133,7 +141,6 @@ def _sample_maintenance(self):
         ########
         # new method to introduce generated maintenance
         self.maintenance = self._generate_maintenance()  #
-
         ##########
         # same as before in GridStateFromFileWithForecasts
         GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
@@ -171,7 +178,12 @@ def _generate_matenance_static(name_line,
                                    daily_proba_per_month_maintenance, 
                                    max_daily_number_per_month_maintenance,
                                    space_prng,
+                                   maintenance_day_of_week=None
                                    ):
+        if maintenance_day_of_week is None:
+            # new in grid2op 1.10.3
+            maintenance_day_of_week = np.arange(5)
+
         # define maintenance dataframe with size (nbtimesteps,nlines)
         columnsNames = name_line
         nbTimesteps = n_
@@ -203,8 +215,6 @@ def _generate_matenance_static(name_line,
         datelist = datelist[:-1]
 
         n_lines_maintenance = len(line_to_maintenance)
-
-        _24_h = timedelta(seconds=86400)
         nb_rows = int(86400 / time_interval.total_seconds())
         selected_rows_beg = int(
             maintenance_starting_hour * 3600 / time_interval.total_seconds()
@@ -220,7 +230,7 @@ def _generate_matenance_static(name_line,
         maxDailyMaintenance = -1
         for nb_day_since_beg, this_day in enumerate(datelist):
             dayOfWeek = this_day.weekday()
-            if dayOfWeek < 5:  # only maintenance starting on working days
+            if dayOfWeek in maintenance_day_of_week:
                 month = this_day.month
 
                 maintenance_me = np.zeros((nb_rows, nb_line_maint))
@@ -279,5 +289,9 @@ def _generate_maintenance(self):
             self.maintenance_ending_hour,
             self.daily_proba_per_month_maintenance,
             self.max_daily_number_per_month_maintenance,
-            self.space_prng
+            self.space_prng,
+            self.maintenance_day_of_week
         )
+
+    def regenerate_with_new_seed(self):
+        self._sample_maintenance()

grid2op/Chronics/fromChronix2grid.py

@@ -309,4 +309,12 @@ def next_chronics(self):
             GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
 
         self.check_validity(backend=None)
+
+    def regenerate_with_new_seed(self):
+        raise ChronicsError("You should not 'cache' the data coming from the "
+                            "`FromChronix2grid`, which is probably why you ended "
+                            "up calling this function. If you want to generate data "
+                            "'on the fly' please do not use the `MultiFolder` or "
+                            "`MultiFolderWithCache` `chronics_class` when making your "
+                            "environment.")
 

grid2op/Chronics/gridValue.py

@@ -856,3 +856,21 @@ def cleanup_action_space(self):
         """
         self.__action_space = None
         # NB the action space is not closed as it is NOT own by this class
+
+    def regenerate_with_new_seed(self):
+        """
+        INTERNAL this function is called by some classes (*eg* :class:`MultifolderWithCache`)
+        when a new seed has been set. 
+        
+        For example, if you use some 'chronics' that generate part of them randomly (*eg* 
+        :class:`GridStateFromFileWithForecastsWithMaintenance`) they need to be aware of this
+        so that a reset actually update the seeds.
+        
+        This is closely related to issue https://github.com/rte-france/Grid2Op/issues/616
+        
+        .. danger::
+            This function should be called only once (not 0, not twice) after a "seed" function has been set. 
+            Otherwise results might not be fully reproducible.
+            
+        """
+        pass

grid2op/Chronics/handlers/baseHandler.py

@@ -494,3 +494,14 @@ def get_init_dict_action(self) -> Union[dict, None]:
         action space.
         """
         raise NotImplementedError()
+
+    def regenerate_with_new_seed(self):
+        """This function is called in case of data being "cached" (for example using the 
+        :class:`grid2op.Chronics.MultifolderWithCache`)
+        
+        In this case, the data in cache needs to be updated if the seed has changed since
+        the time they have been added to it.
+        
+        If your handler has some random part, we recommend you to implement this function.
+        Otherwise feel free to ignore it"""
+        pass

grid2op/Chronics/handlers/jsonMaintenanceHandler.py

@@ -63,7 +63,8 @@ def __init__(self,
         self.n_line = None  # used in one of the GridStateFromFileWithForecastsWithMaintenance functions
         self._duration_episode_default = _duration_episode_default
         self.current_index = 0
-
+        self._order_backend_arrays = None
+
     def get_maintenance_time_1d(self, maintenance):
         return GridValue.get_maintenance_time_1d(maintenance)
 
@@ -82,7 +83,8 @@ def _create_maintenance_arrays(self, current_datetime):
             self.dict_meta_data["maintenance_ending_hour"],
             self.dict_meta_data["daily_proba_per_month_maintenance"],
             self.dict_meta_data["max_daily_number_per_month_maintenance"],
-            self.space_prng
+            self.space_prng,
+            self.dict_meta_data["maintenance_day_of_week"] if "maintenance_day_of_week" in self.dict_meta_data else None
         )
         GridStateFromFileWithForecastsWithMaintenance._fix_maintenance_format(self)
 
@@ -128,4 +130,8 @@ def _clear(self):
 
     def done(self):
         # maintenance can be generated on the fly so they are never "done"
-        return False
+        return False
+
+    def regenerate_with_new_seed(self):
+        if self.dict_meta_data is not None:
+            self._create_maintenance_arrays(self.init_datetime)

grid2op/Chronics/handlers/noisyForecastHandler.py

@@ -212,3 +212,7 @@ def forecast(self,
                     res *= self._env_loss_ratio(inj_dict_env)
                 # TODO ramps, pmin, pmax !
         return res.astype(dt_float) if res is not None else None
+
+    def regenerate_with_new_seed(self):
+        # there is nothing to do for this handler as things are generated "on the fly"
+        pass