Allow parameter modification from an input file for restarts (parthen…

…on-hpc-lab#1004)

* Allow parameter modification from an input file for restarts

* Fix typos

* Test override from input for restart

* Fix path to restart override

CHANGELOG.md

@@ -3,6 +3,36 @@
 ## Current develop
 
 ### Added (new features/APIs/variables/...)
+
+
+### Changed (changing behavior/API/variables/...)
+- [[PR 1024]](https://github.com/parthenon-hpc-lab/parthenon/pull/1024) Add .outN. to history output filenames
+- [[PR 1004]](https://github.com/parthenon-hpc-lab/parthenon/pull/1004) Allow parameter modification from an input file for restarts
+
+### Fixed (not changing behavior/API/variables/...)
+- [[PR 1024]](https://github.com/parthenon-hpc-lab/parthenon/pull/1024) Add features to history output
+- [[PR 1031]](https://github.com/parthenon-hpc-lab/parthenon/pull/1031) Fix bug in non-cell centered AMR
+
+### Infrastructure (changes irrelevant to downstream codes)
+- [[PR 1009]](https://github.com/parthenon-hpc-lab/parthenon/pull/1009) Move from a single octree to a forest of octrees
+
+
+### Removed (removing behavior/API/varaibles/...)
+
+
+### Incompatibilities (i.e. breaking changes)
+
+
+## Release 24.03
+Date: 2024-03-21
+
+### Added (new features/APIs/variables/...)
+- [[PR 852]](https://github.com/parthenon-hpc-lab/parthenon/pull/852) Add Mesh version of UserWorkBeforeOutput
+- [[PR 998]](https://github.com/parthenon-hpc-lab/parthenon/pull/998) tensor indices added to sparse pack
+- [[PR 999]](https://github.com/parthenon-hpc-lab/parthenon/pull/999) Add a post-initialization hook
+- [[PR 987]](https://github.com/parthenon-hpc-lab/parthenon/pull/987) New tasking infrastructure and capabilities
+- [[PR 969]](https://github.com/parthenon-hpc-lab/parthenon/pull/969) New macro-based auto-naming of profiling regions and kernels
+- [[PR 981]](https://github.com/parthenon-hpc-lab/parthenon/pull/981) Add IndexSplit
 - [[PR 983]](https://github.com/parthenon-hpc-lab/parthenon/pull/983) Add Contains to SparsePack
 - [[PR 968]](https://github.com/parthenon-hpc-lab/parthenon/pull/968) Add per package registration of boundary conditions
 - [[PR 948]](https://github.com/parthenon-hpc-lab/parthenon/pull/948) Add solver interface and update Poisson geometric multi-grid example

doc/sphinx/src/README.rst

@@ -195,13 +195,13 @@ pointers in the ApplicationInput member app_input of the
 ParthenonManager class prior to calling ``ParthenonInit``. This is
 demonstrated in the ``main()`` functions in the examples.
 
-Note that the ``ProblemGenerator``\ s of ``Mesh`` and ``MeshBlock`` are
-mutually exclusive. Moreover, the ``Mesh`` one requires
-``parthenon/mesh/pack_size=-1`` during initialization, i.e., all blocks
-on a rank need to be in a single pack. This allows to use MPI reductions
-inside the function, for example, to globally normalize quantities. The
-``parthenon/mesh/pack_size=-1`` exists only during problem
-inititalization, i.e., simulations can be restarted with an arbitrary
+Note that the ``ProblemGenerator``\ s (and ``PostInitialization``\ s) of
+``Mesh`` and ``MeshBlock`` are mutually exclusive. Moreover, the ``Mesh``
+ones requires ``parthenon/mesh/pack_size=-1`` during initialization, i.e.,
+all blocks on a rank need to be in a single pack. This allows to use MPI
+reductions inside the function, for example, to globally normalize quantities.
+The ``parthenon/mesh/pack_size=-1`` exists only during problem
+initialization, i.e., simulations can be restarted with an arbitrary
 ``pack_size``. For an example of the ``Mesh`` version, see the `Poisson
 example <https://github.com/parthenon-hpc-lab/parthenon/blob/develop/example/poisson/parthenon_app_inputs.cpp>`__.
 
@@ -303,7 +303,12 @@ command. For example, the ``refine_tol`` parameter in the
 appending ``parthenon/refinement0/refine_tol=my_new_value`` to the
 launch command (e.g.,
 ``srun ./myapp -i my_input.file parthenon/refinement0/refine_tol=my_new_value``).
-This similarly applies to simulations that are restarted.
+This similarly applies to simulations that are restarted, where modifications
+from the command line and through an input file are possible, e.g.,
+``srun ./myapp -r out0.rhdf -i modified_input.in parthenon/refinement0/refine_tol=my_new_value``.
+In the latter case, the input stored in the restart file will be read first,
+then updated from the content in the input file, and finally modified from the
+parameters provided on the command line.
 
 Global reductions
 ~~~~~~~~~~~~~~~~~

doc/sphinx/src/outputs.rst

@@ -3,10 +3,10 @@
 Outputs
 =======
 
-Outputs from Parthenon are controled via ``<parthenon/output*>`` blocks,
+Outputs from Parthenon are controlled via ``<parthenon/output*>`` blocks,
 where ``*`` should be replaced by a unique integer for each block.
 
-To disable an output block without removing it from the intput file set
+To disable an output block without removing it from the input file set
 the block's ``dt < 0.0``.
 
 In addition to time base outputs, two additional options to trigger
@@ -113,7 +113,7 @@ Restart Files
 
 Parthenon allows users to output restart files for restarting a
 simulation. The restart file captures the input file, so no input file
-is required to be specified. Parameters for the input can be overriden
+is required to be specified. Parameters for the input can be overridden
 in the usual way from the command line. At a future date we will allow
 for users the ability to extensively edit the parameters stored within
 the restart file.
@@ -133,8 +133,9 @@ This will produce an hdf5 (``.rhdf``) output file every 1 units of
 simulation time that can be used for restarting the simulation.
 
 To use this restart file, simply specify the restart file with a
-``-r <restart.rhdf>`` at the command line. It is an error to specify an
-input file with the ``-i`` flag when using the restart option.
+``-r <restart.rhdf>`` at the command line. If both ``-r <restart.rhdf>``
+and ``-i <input.in>`` are specified, the simulation will be restarted from
+the restart file with input parameters updated (or added) from the input file.
 
 For physics developers: The fields to be output are automatically
 selected as all the variables that have either the ``Independent`` or
@@ -174,7 +175,7 @@ Currently supported are
 - weighting by volume and/or variable
 
 The output format follows ``numpy`` convention, so that plotting data
-with Python based machinery should be straightfoward (see example below).
+with Python based machinery should be straightforward (see example below).
 In other words, 2D histograms use C-ordering corresponding to ``[x,y]``
 indexing with ``y`` being the fast index.
 In general, histograms are calculated using inclusive left bin edges and
@@ -217,7 +218,7 @@ with the following parameters
 
 - ``hist_names=STRING, STRING, STRING, ...`` (comma separated names)
    The names of the histograms in this block.
-   Will be used as preifx in the block as well as in the output file.
+   Will be used as prefix in the block as well as in the output file.
    All histograms will be written to the same output file with the "group" in the
    output corresponding to the histogram name.
 - ``NAME_ndim=INT`` (either ``1`` or ``2``)
@@ -333,7 +334,7 @@ Support for Ascent is disabled by default and must be enabled via ``PARTHENON_EN
 In the input file, include a ``<parthenon/output*>`` block and specify ``file_type = ascent``.
 A ``dt`` parameter controls the frequency of outputs for simulations involving evolution.
 *Note* that in principle Ascent can control its own output cadence (including
-automated tiggers).
+automated triggers).
 If you want to call Ascent on every cycle, set ``dt`` to a value smaller than the actual simulation ``dt``.
 The mandatory ``actions_file`` parameter points to a separate file that defines
 Ascent actions in ``.yaml`` or ``.json`` format, see
@@ -347,7 +348,7 @@ If component label(s) are provided, they will be added as a suffix, e.g,.
 Otherwise, an integer index is added for vectors/tensors with more than one component, i.e.,
 vectors/tensors with a single component and without component labels will not contain a suffix.
 The definition of component labels for variables is typically done by downstream codes
-so that the downstream documention should be consulted for more specific information.
+so that the downstream documentation should be consulted for more specific information.
 
 A ``<parthenon/output*>`` block might look like::
 

src/argument_parser.hpp

@@ -47,7 +47,6 @@ class ArgParse {
         case 'i': // -i <input_filename>
           invalid = invalid_arg();
           input_filename = argv[++i];
-          iarg_flag = 1;
           break;
         case 'r': // -r <restart_file>
           invalid = invalid_arg();
@@ -129,7 +128,6 @@ class ArgParse {
   int analysis_flag = 0;
   int res_flag = 0;
   int narg_flag = 0;
-  int iarg_flag = 0;
   int mesh_flag = 0;
   int wtlim = 0;
   int exit_flag = 0;

src/parthenon_manager.cpp

@@ -95,10 +95,9 @@ ParthenonStatus ParthenonManager::ParthenonInitEnv(int argc, char *argv[]) {
   SignalHandler::SignalHandlerInit();
   if (Globals::my_rank == 0 && arg.wtlim > 0) SignalHandler::SetWallTimeAlarm(arg.wtlim);
 
-  // Populate the ParameterInput object
-  if (arg.input_filename != nullptr) {
-    pinput = std::make_unique<ParameterInput>(arg.input_filename);
-  } else if (arg.res_flag != 0) {
+  // Populate the ParameterInput object.
+  // If restart, then ParameterInput in the restart file takes precedence.
+  if (arg.res_flag != 0) {
     // Read input from restart file
     restartReader = std::make_unique<RestartReader>(arg.restart_filename);
 
@@ -108,6 +107,20 @@ ParthenonStatus ParthenonManager::ParthenonInitEnv(int argc, char *argv[]) {
     std::istringstream is(inputString);
     pinput->LoadFromStream(is);
   }
+  // If an input was provided
+  if (arg.input_filename != nullptr) {
+    // Modify info read from restart file
+    if (arg.res_flag != 0) {
+      IOWrapper infile;
+      infile.Open(arg.input_filename, IOWrapper::FileMode::read);
+      pinput->LoadFromFile(infile);
+      infile.Close();
+
+      // Populate new object for fresh simulation
+    } else {
+      pinput = std::make_unique<ParameterInput>(arg.input_filename);
+    }
+  }
 
   // Modify based on command line inputs
   pinput->ModifyFromCmdline(argc, argv);

tst/regression/test_suites/restart/parthinput_override.restart

@@ -0,0 +1,9 @@
+# ========================================================================================
+# Parthenon performance portable AMR framework
+# Copyright(C) 2024 The Parthenon collaboration
+# Licensed under the 3-clause BSD License, see LICENSE file for details
+# ========================================================================================
+
+# Testing to override parameters in a restart file from an input file
+<parthenon/job>
+problem_id=silver

tst/regression/test_suites/restart/restart.py

@@ -1,6 +1,6 @@
 # ========================================================================================
 # Parthenon performance portable AMR framework
-# Copyright(C) 2020-2021 The Parthenon collaboration
+# Copyright(C) 2020-2024 The Parthenon collaboration
 # Licensed under the 3-clause BSD License, see LICENSE file for details
 # ========================================================================================
 # (C) (or copyright) 2020-2021. Triad National Security, LLC. All rights reserved.
@@ -38,7 +38,8 @@ def Prepare(self, parameters, step):
             parameters.driver_cmd_line_args = [
                 "-r",
                 "gold.out0.00001.rhdf",
-                "parthenon/job/problem_id=silver",
+                "-i",
+                f"{parameters.parthenon_path}/tst/regression/test_suites/restart/parthinput_override.restart",
                 "-t",
                 "00:00:02",
             ]