From 11e2f5cb496f2409b113b9882ff952605e842d32 Mon Sep 17 00:00:00 2001 From: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> Date: Fri, 26 Jan 2024 11:14:22 -0500 Subject: [PATCH] US-157: Edit uwtools documentation (#399) Minor edits for grammar, formatting, etc. --- docs/conf.py | 2 + docs/index.rst | 20 +++--- .../contributor_guide/code_quality.rst | 6 +- .../contributor_guide/developer_setup.rst | 8 +-- .../contributor_guide/documentation.rst | 12 ++-- .../contributor_guide/fork_pr_model.rst | 2 +- docs/sections/user_guide/cli/mode_config.rst | 68 ++++++++++--------- docs/sections/user_guide/cli/mode_rocoto.rst | 14 ++-- .../sections/user_guide/cli/mode_template.rst | 34 +++++----- docs/sections/user_guide/installation.rst | 12 ++-- .../user_guide/uw_yaml/rocoto_yaml.rst | 42 +++++++----- docs/shared/miniforge3_instructions.rst | 4 +- 12 files changed, 120 insertions(+), 104 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index ceb7cf95d..c56003d01 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -24,6 +24,7 @@ project = "Unified Workflow Tools" release = _metadata["version"] version = _metadata["version"] +user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" extlinks = { "anaconda": ("https://www.anaconda.com/%s", "%s"), @@ -44,6 +45,7 @@ "miniforge": ("https://github.com/conda-forge/miniforge/%s", "%s"), "miniforge3": ("https://github.com/conda-forge/miniforge/%s", "%s"), "mypy": ("https://mypy.readthedocs.io/en/stable/%s", "%s"), + "noaa": ("https://www.noaa.gov/%s", "%s"), "pylint": ("https://pylint.readthedocs.io/en/v3.0.3/%s", "%s"), "pytest": ("https://docs.pytest.org/en/7.4.x/%s", "%s"), "rocoto": ("https://christopherwharrop.github.io/rocoto/%s", "%s"), diff --git a/docs/index.rst b/docs/index.rst index 3fb633003..e56502227 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -7,20 +7,20 @@ Unified Workflow Tools ====================== -``uwtools`` is an open-source modern Python package that helps automate common tasks needed for many standard numerical weather prediction (NWP) workflows and provides drivers to automate the configuration and execution of :ufs:`Unified Forecast System (UFS)<>` components, providing flexibility, interoperability, and usability to various UFS Applications. +``uwtools`` is a modern, open-source Python package that helps automate common tasks needed for many standard numerical weather prediction (NWP) workflows. It also provides drivers to automate the configuration and execution of :ufs:`Unified Forecast System (UFS)<>` components, providing flexibility, interoperability, and usability to various UFS Applications. If you're interested in contributing, check out the :doc:`Contributor Guide `. For users who want to start using the toolbox and framework, take a peek at our :doc:`User Guide `. -.. contents:: On this page +.. contents:: On This Page: :depth: 2 :local: The Tools --------- -The tools are accessible from both a command-line interface (CLI) and a Python API. The CLI automates many operations commonly needed in NWP workflows. The API supports all CLI operations, and additionally provides access to in-memory objects to facilitate more novel use cases. We hope these options will let you integrate the package into your pre-existing bash and Python scripts, and give you some handy tools to use in your day-to-day work with running NWP systems. +The tools are accessible from both a command-line interface (CLI) and a Python API. The CLI automates many operations commonly needed in NWP workflows. The API supports all CLI operations and additionally provides access to in-memory objects to facilitate more novel use cases. We hope these options will let you integrate the package into your pre-existing bash and Python scripts, and give you some handy tools to use in your day-to-day work with running NWP systems. Configuration Management ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -28,12 +28,12 @@ Configuration Management | **CLI**: ``uw config -h`` | **API**: ``import uwtools.api.config`` -The config tool suite helps you compare, transform, modify, and even validate your configuration. The package supports YAML, shell, Fortran namelist, and INI file formats. Configuration in any of these formats may express values using :jinja2:`Jinja2 syntax`. These values can reference others, or compute new values by evaluating mathematical expressions, building paths, manipulating strings, etc. +The config tool suite helps you compare, transform, modify, and even validate your configuration. The package supports YAML, shell, Fortran namelist, and INI file formats. Configuration in any of these formats may use :jinja2:`Jinja2 syntax` to express values. These values can reference others, or compute new values by evaluating mathematical expressions, building paths, manipulating strings, etc. Compare Mode """""""""""" -When the Linux diff tool just doesn't work for comparing unordered namelists with mixed-case keys, this is your go-to! It also works on the other configuration formats, but the Fortran namelists are the *real* catalyst behind this gem! +When the Linux diff tool just doesn't work for comparing unordered namelists with mixed-case keys, this is your go-to! The Fortran namelists are the *real* catalyst behind this gem, but it also works on the other configuration formats. | :any:`CLI documentation with examples` @@ -67,7 +67,7 @@ The ``render`` mode that gives you the full power of rendering a :jinja2:`Jinja2 Translate Mode """""""""""""" -This tool helps transform legacy configuration files templated with the atparse tool (common at NOAA) into :jinja2:`Jinja2 templates` for use with the ``uw config realize`` and ``uw template render`` tools, or their API equivalents. +This tool helps transform legacy configuration files templated with the atparse tool (common at :noaa:`NOAA<>`) into :jinja2:`Jinja2 templates` for use with the ``uw config realize`` and ``uw template render`` tools, or their API equivalents. | :any:`CLI documentation with examples` @@ -82,14 +82,14 @@ This tool is all about creating a configurable interface to the :rocoto:`Rocoto< Realize Mode """""""""""" -This is where you put in your structured YAML that defines your workflow of choice and it pops out a verified Rocoto XML. +This is where you put in your structured YAML that defines your workflow of choice, and it pops out a verified Rocoto XML. | :any:`CLI documentation with examples` Validate Mode """"""""""""" -Do you already have a Rocoto XML, but don't want to run Rocoto to make sure it works? Use the validate mode to check to see if Rocoto will be happy. +Do you already have a Rocoto XML but don't want to run Rocoto to make sure it works? Use the validate mode to check to see if Rocoto will be happy. | :any:`CLI documentation with examples` @@ -104,9 +104,9 @@ Forecast | **CLI**: ``uw forecast -h`` | **API**: ``import uwtools.api.drivers.forecast`` -This driver is the first of its kind (with many others to come) and takes a few pieces of information from the user – the model, the time, and a structured YAML – and it runs a forecast via batch job or as an executable. That simple. +This driver is the first of its kind (with many others to come) and takes a few pieces of information from the user --- the model, the time, and a structured YAML --- and runs a forecast via batch job or as an executable. That simple. -We've helped by providing a JSON Schema that allows you to validate your YAML to ensure you’ve got it right! +We've helped by providing a JSON Schema that allows you to validate your YAML to ensure you've got it right! Over time, we'll add many other drivers to support a variety of UFS components from pre-processing to post-processing, along with many data assimilation components. diff --git a/docs/sections/contributor_guide/code_quality.rst b/docs/sections/contributor_guide/code_quality.rst index 9426df945..64b047148 100644 --- a/docs/sections/contributor_guide/code_quality.rst +++ b/docs/sections/contributor_guide/code_quality.rst @@ -24,11 +24,11 @@ Configuration for these tools is provided by the file ``src/pyproject.toml``. The order of the targets above is intentional: - * ``make format`` will complain about certain kinds of syntax errors that would cause all the remaining code-quality tools to fail (and may change line numbers reported by other tools, if it ran after them). + * ``make format`` will complain about certain kinds of syntax errors that would cause all the remaining code-quality tools to fail (and that could change line numbers reported by other tools, if it ran after them). * ``make lint`` provides a good first check for obvious errors and anti-patterns in the code. * ``make typecheck`` offers a more nuanced look at interfaces between functions, methods, etc. and may spot issues missed by the linter. * ``make unittest`` provides higher-level semantic-correctness checks once code syntax and typing is deemed correct. -All the above tests are executed by the CI system when code is merged to specific git branches, and again when a conda package is built for release. To ensure that these processes succeed, be sure to run all the tests in a development shell before opening a pull request, and throughout the PR's lifecycle as subsequent changes are made. CI will reject unformatted code, so also run ``make format`` and commit any changes it makes. A useful development idiom is to periodically run ``make format && make test`` to perform a full code-quality sweep through the code. +All the above tests are executed by the CI system when code is merged to specific git branches and when a conda package is built for release. To ensure that these processes succeed, be sure to run all the tests in a development shell before opening a pull request, and throughout the PR's lifecycle as subsequent changes are made. CI will reject unformatted code, so also run ``make format`` and commit any changes it makes. A useful development idiom is to periodically run ``make format && make test`` to perform a full code-quality sweep through the code. -The ``uwtools`` repository has standardized on 100% unit-test coverage, enforced by ``make unittest`` and its configuration in ``pyproject.toml``. Please help maintain this high standard. +The ``uwtools`` repository has standardized 100% unit-test coverage, enforced by ``make unittest`` and its configuration in ``pyproject.toml``. Please help maintain this high standard. diff --git a/docs/sections/contributor_guide/developer_setup.rst b/docs/sections/contributor_guide/developer_setup.rst index f87dc5c21..37adb3b2a 100644 --- a/docs/sections/contributor_guide/developer_setup.rst +++ b/docs/sections/contributor_guide/developer_setup.rst @@ -23,7 +23,7 @@ If an existing conda (:miniforge:`Miniforge<>`, :miniconda:`Miniconda<>`, :anaco If the above is successful, you will be in a ``uwtools`` development shell. See below for usage information. You may exit the shell with ``exit`` or ``ctrl-d``. -Future ``make devshell`` invocations will be almost instantaneous, as the underlying virtual environment will already exist. In general, all source code changes will be immediately live in the development shell, subject to execution, test, etc. But some changes – especially to the contents of the ``recipe/`` directory, or to the ``src/setup.py`` module – may require recreation of the development shell. If you know this is needed, or when in doubt: +Future ``make devshell`` invocations will be almost instantaneous, as the underlying virtual environment will already exist. In general, all source code changes will be immediately live in the development shell, subject to execution, test, etc. But some changes --- especially to the contents of the ``recipe/`` directory or to the ``src/setup.py`` module --- may require recreation of the development shell. If you know this is needed, or when in doubt: #. Exit the development shell. #. Run ``make clean-devenv`` (or ``conda env remove -n DEV-uwtools``) to remove the old environment. @@ -42,7 +42,7 @@ Using a ``bash`` Development Shell A development shell makes available several code-formatting and quality checkers, which should be periodically run during the development process. See :doc:`Code Quality ` for full details. -In addition to the ``make devshell`` command, other ``make`` targets are available for use *outside* a development shell, i.e. from the base conda environment (requires presence of the ``condev`` package): +In addition to the ``make devshell`` command, other ``make`` targets are available for use *outside* a development shell, i.e., from the base conda environment (requires presence of the ``condev`` package): .. list-table:: :widths: 15 85 @@ -62,7 +62,7 @@ These targets work from the code in its current state in the clone. ``make env`` Building ``condev`` Locally --------------------------- -As an alternative to installing :anaconda-condev:`pre-built package<>`, the ``condev`` package can be built locally, then installed into the local conda installation. Ensure that ``conda-build`` and ``conda-verify`` are installed in the base environment: +As an alternative to installing the :anaconda-condev:`pre-built package<>`, the ``condev`` package can be built locally, then installed into the local conda installation. Ensure that ``conda-build`` and ``conda-verify`` are installed in the base environment: .. code-block:: text @@ -74,7 +74,7 @@ As an alternative to installing :anaconda-condev:`pre-built package<>`, the ``co Files Derived from ``condev`` ----------------------------- -The following files in this repo are derived from their counterparts in the :condev:`condev demo` and are used by ``condev`` code when running certain make commands +The following files in this repository are derived from their counterparts in the :condev:`condev demo` and are used by ``condev`` code when running certain make commands .. code-block:: text diff --git a/docs/sections/contributor_guide/documentation.rst b/docs/sections/contributor_guide/documentation.rst index 61e3f3dc8..800f0b950 100644 --- a/docs/sections/contributor_guide/documentation.rst +++ b/docs/sections/contributor_guide/documentation.rst @@ -17,16 +17,16 @@ The ``make docs`` command will build the docs under ``docs/build/html``, after w file:///docs/build/html/index.index.html -Re-run ``make docs`` and refresh your browser after making and saving changes. +Rerun ``make docs`` and refresh your browser after making and saving changes. -If, at some point, you remove and recreate the conda development environment underlying your development shell, you will need to re-run the ``source install-deps`` command in the new environment/shell. Until then, the installed doc packages will persist and support docs generation. +If, at some point, you remove and recreate the conda development environment underlying your development shell, you will need to rerun the ``source install-deps`` command in the new environment/shell. Until then, the installed doc packages will persist and support docs generation. Viewing Online Documentation ---------------------------- -Online documentation generation and hosting for ``uwtools`` is provided by :rtd:`Read the Docs<>`. The green *View Docs* button near the upper right of that page links to the official docs for the project, as built for the ``main`` branch. Docs are also generated for the ``develop`` branch, and can be selected by changing **v: main** to **v: develop** via the small green pull-down menu located at the bottom of the navigation pane on the left of the page. +Online documentation generation and hosting for ``uwtools`` is provided by :rtd:`Read the Docs<>`. The green *View Docs* button near the upper right of that page links to the official docs for the project, as built for the ``main`` branch. Docs are also generated for the ``develop`` branch and can be selected by changing **v: main** to **v: develop** via the small green pull-down menu located at the bottom of the navigation pane on the left of the page. -Docs are also built and published when Pull Requests (PRs) targeting the ``develop`` or ``main`` branches are opened. Visit the :rtd:`Builds page` to see recent builds, including those made for PRs. Click a PR-related build marked *Passed*, then the small *View docs* link (**not** the large green *View Docs* button) to see the docs built specifically for that PR. If your PR includes documentation updates, it may be helpful to include the URL of this build in your PR's description so that reviewers can see the rendered HTML docs and not just the modified ``.rst`` files. Note that, if commits are pushed to the PR's source branch, Read the Docs will rebuild the PR docs: See the checks section near the bottom of a PR for current status, and for another link to the PR docs via the *Details* link. +Docs are also built and published when Pull Requests (PRs) targeting the ``develop`` or ``main`` branches are opened. Visit the :rtd:`Builds page` to see recent builds, including those made for PRs. Click a PR-related build marked *Passed*, then the small *View docs* link (**not** the large green *View Docs* button) to see the docs built specifically for that PR. If your PR includes documentation updates, it may be helpful to include the URL of this build in your PR's description so that reviewers can see the rendered HTML docs and not just the modified ``.rst`` files. Note that if commits are pushed to the PR's source branch, Read the Docs will rebuild the PR docs. See the checks section near the bottom of a PR for current status, and for another link to the PR docs via the *Details* link. Documentation Guidelines ------------------------ @@ -44,9 +44,9 @@ Please follow these guidelines when contributing to the documentation: * The synopsis information printed by ``uw [mode [action]] --help`` is automatically wrapped and indented based on current terminal size. For visual consistency, please set your terminal width to 100 columns when running such commands to produce output to copy into the docs. * Follow the :rst:`RST Sections` guidelines, underlining section headings with = characters, subsections with - characters, and subsubsections with ^ characters. If a further level of refinement is needed, use " to underline paragraph headers. * In [[sub]sub]section titles, capitalize all "principal" words. In practice this usually means all words but articles (a, an, the), logicals (and, etc.), and prepositions (for, of, etc.). Always fully capitalize acronyms (e.g., YAML). -* Never capitalize proper names when their owners do not (e.g. write `"pandas" `_, not "Pandas", even at the start of a sentence) or when referring to a software artifact (e.g. write ``numpy`` when referring to the library, and "NumPy" when referring to the project). +* Never capitalize proper names when their owners do not (e.g., write `"pandas" `_, not "Pandas", even at the start of a sentence) or when referring to a software artifact (e.g., write ``numpy`` when referring to the library, and "NumPy" when referring to the project). * When referring to YAML constructs, `block` refers to an entry whose values is a nested collection of key/value pairs, while `entry` is a single key/value pair. -* When using the ``.. code-block::`` directive, align the actual code with the word ``code``. Also, when ``.. code-block::`` directives appear in bulleted or numberd lists, align them with the text following the space to the righ of the bullet/number. For example: +* When using the ``.. code-block::`` directive, align the actual code with the word ``code``. Also, when ``.. code-block::`` directives appear in bulleted or numberd lists, align them with the text following the space to the right of the bullet/number. For example: .. code-block:: text diff --git a/docs/sections/contributor_guide/fork_pr_model.rst b/docs/sections/contributor_guide/fork_pr_model.rst index 37a4e8b87..3a3ffa400 100644 --- a/docs/sections/contributor_guide/fork_pr_model.rst +++ b/docs/sections/contributor_guide/fork_pr_model.rst @@ -12,7 +12,7 @@ Contributions to the ``uwtools`` project are made via a :github-docs:`Fork` in your clone / to your fork. (Refer to the :doc:`Developer Setup ` page for setting up a development shell, formatting and testing your code, etc.) #. When your work is complete, :github-docs:`create a pull request` to merge your changes. -For future contributions, you may delete and then re-create your fork, or configure the official ``uwtools`` repo as :github-docs:`remote repository` on your clone and :github-docs:`sync upstream changes` to stay up-to-date with the official repo. +For future contributions, you may delete and then recreate your fork or configure the official ``uwtools`` repository as a :github-docs:`remote repository` on your clone and :github-docs:`sync upstream changes` to stay up-to-date with the official repository. Specifics for ``uwtools`` ------------------------- diff --git a/docs/sections/user_guide/cli/mode_config.rst b/docs/sections/user_guide/cli/mode_config.rst index 122ce3142..cc52f806e 100644 --- a/docs/sections/user_guide/cli/mode_config.rst +++ b/docs/sections/user_guide/cli/mode_config.rst @@ -1,7 +1,7 @@ Mode ``config`` =============== -The ``uw`` mode for handling configs. +The ``uw`` mode for handling configuration files (configs). .. code-block:: text @@ -28,6 +28,8 @@ The ``uw`` mode for handling configs. ``compare`` ----------- +The ``compare`` mode lets users compare two config files. + .. code-block:: text $ uw config compare --help @@ -60,7 +62,7 @@ The ``uw`` mode for handling configs. Examples ^^^^^^^^ -The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, both initially with content +The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, both initially with the following contents: .. code-block:: fortran @@ -69,7 +71,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, recipient = "World" / -* Compare two config files with the same contents: +* To compare two config files with the same contents: .. code-block:: text @@ -88,7 +90,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, [2024-01-08T16:54:03] INFO --------------------------------------------------------------------- [2024-01-08T16:54:03] INFO values: recipient: - None + World -* If a config file has an unrecognized (or no) extension, ``uw`` will not know how to parse its content: +* If a config file has an unrecognized (or no) extension, ``uw`` will not know how to parse its contents: .. code-block:: text @@ -105,7 +107,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, [2024-01-08T16:56:54] INFO --------------------------------------------------------------------- [2024-01-08T16:56:54] INFO values: recipient: - None + World -* Request verbose log output: +* To request verbose log output: .. code-block:: text @@ -124,7 +126,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, $ uw config compare --file-1-path values1.nml --file-2-path values2.nml --verbose 2>compare.log - The content of ``compare.log``: + The contents of ``compare.log``: .. code-block:: text @@ -146,6 +148,8 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, ``realize`` ----------- +In ``uw`` terminology, to realize a configuration file is to transform it from its raw form into its final, usable state. The ``realize`` mode can build a complete config file from two or more separate files. + .. code-block:: text $ uw config realize --help @@ -183,7 +187,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``, Examples ^^^^^^^^ -The examples that follow use YAML file ``config.yaml`` with content +The examples in this section use the YAML file ``config.yaml`` with the following contents: .. code-block:: yaml @@ -195,7 +199,7 @@ The examples that follow use YAML file ``config.yaml`` with content recipient: World repeat: 1 -supplemental YAML file ``values1.yaml`` with content +and the supplemental YAML file ``values1.yaml`` with the following contents: .. code-block:: yaml @@ -205,7 +209,7 @@ supplemental YAML file ``values1.yaml`` with content recipient: Moon repeat: 2 -and additional supplemental YAML file ``values2.yaml`` with content +and an additional supplemental YAML file ``values2.yaml`` with the following contents: .. code-block:: yaml @@ -213,7 +217,7 @@ and additional supplemental YAML file ``values2.yaml`` with content empty: false repeat: 3 -* Show the values in the input config file that have unrendered Jinja2 variables/expressions or empty keys: +* To show the values in the input config file that have unrendered Jinja2 variables/expressions or empty keys: .. code-block:: text @@ -244,9 +248,9 @@ and additional supplemental YAML file ``values2.yaml`` with content recipient: World repeat: 1 - Shell redirection via ``|``, ``>``, et al may also be used to stream output to a file, another process, etc. + Shell redirection via ``|``, ``>``, et al. may also be used to stream output to a file, another process, etc. -* Values in the primary input file can be overridden via one or more supplemental files specified as positional arguments, each overriding the last; or by environment variables, which have the highest precedence. +* Values in the primary input file can be overridden via one or more supplemental files specified as positional arguments, each overriding the last, or by environment variables, which have the highest precedence. .. code-block:: text @@ -259,13 +263,13 @@ and additional supplemental YAML file ``values2.yaml`` with content recipient: Moon repeat: 3 -* Realize the config to a file via command-line argument: +* To realize the config to a file via command-line argument: .. code-block:: text $ uw config realize --input-file config.yaml --output-file realized.yaml values1.yaml - The content of ``realized.yaml``: + The contents of ``realized.yaml``: .. code-block:: yaml @@ -290,14 +294,14 @@ and additional supplemental YAML file ``values2.yaml`` with content [2024-01-23T22:31:08] INFO recipient: Moon [2024-01-23T22:31:08] INFO repeat: 2 -* If an input file is read alone from ``stdin``, ``uw`` will not know how to parse its content: +* If an input file is read alone from ``stdin``, ``uw`` will not know how to parse its contents: .. code-block:: text $ cat config.yaml | uw config realize --output-file realized.yaml values1.yaml Specify --input-format when --input-file is not specified -* Read the config from ``stdin`` and realize to ``stdout``: +* To read the config from ``stdin`` and realize to ``stdout``: .. code-block:: text @@ -310,7 +314,7 @@ and additional supplemental YAML file ``values2.yaml`` with content recipient: Moon repeat: 2 -* If the config file has an unrecognized (or no) extension, ``uw`` will not know how to parse its content: +* If the config file has an unrecognized (or no) extension, ``uw`` will not know how to parse its contents: .. code-block:: text @@ -330,7 +334,7 @@ and additional supplemental YAML file ``values2.yaml`` with content recipient: Moon repeat: 2 -* Request verbose log output: +* To request verbose log output: .. code-block:: text @@ -370,7 +374,7 @@ and additional supplemental YAML file ``values2.yaml`` with content $ uw config realize --input-file config.yaml --output-format yaml --verbose values1.yaml >realized.yaml 2>realized.log - The content of ``realized.yaml``: + The contents of ``realized.yaml``: .. code-block:: yaml @@ -382,7 +386,7 @@ and additional supplemental YAML file ``values2.yaml`` with content recipient: Moon repeat: 2 - The content of ``realized.log``: + The contents of ``realized.log``: .. code-block:: text @@ -419,7 +423,7 @@ and additional supplemental YAML file ``values2.yaml`` with content .. note:: Combining configs with incompatible depths is not supported. ``ini`` and ``nml`` configs are depth-2, as they organize their key-value pairs (one level) under top-level sections or namelists (a second level). ``sh`` configs are depth-1, and ``yaml`` configs have arbitrary depth. - For example, when attempting to generate an ``sh`` config from a depth-2 ``yaml``: + For example, when attempting to generate a ``sh`` config from a depth-2 ``yaml``: .. code-block:: text @@ -429,8 +433,8 @@ and additional supplemental YAML file ``values2.yaml`` with content .. note:: In recognition of the different sets of value types representable in each config format, ``uw`` supports two format-combination schemes: - 1. Output matches input: The format of the output config matches that of the input config. - 2. Input is YAML: If the input config is YAML, any output format may be requested. In the worst case, values always have a string representation; but note that, for example, the string representation of a YAML sequence (Python ``list``) in an INI output config may not be useful. + 1. **Output matches input:** The format of the output config matches that of the input config. + 2. **Input is YAML:** If the input config is YAML, any output format may be requested. In the worst case, values always have a string representation, but note that, for example, the string representation of a YAML sequence (Python ``list``) in an INI output config may not be useful. In all cases, any supplemental configs must be in the same format as the input config and must have recognized extensions. @@ -451,6 +455,8 @@ and additional supplemental YAML file ``values2.yaml`` with content ``validate`` ------------ +The ``validate`` mode ensures that a given config file is structured properly. + .. code-block:: text $ uw config validate --help @@ -477,7 +483,7 @@ and additional supplemental YAML file ``values2.yaml`` with content Examples ^^^^^^^^ -The examples that follow use :json-schema:`JSON Schema` file ``schema.jsonschema`` with content +The examples that follow use the :json-schema:`JSON Schema` file ``schema.jsonschema`` with the following contents: .. code-block:: json @@ -503,7 +509,7 @@ The examples that follow use :json-schema:`JSON Schema``, et al may also be used to stream output to a file, another process, etc. + Shell redirection via ``|``, ``>``, et al. may also be used to stream output to a file, another process, etc. -* Read the config from ``stdin`` and print validation results to ``stdout``: +* To read the *config* from ``stdin`` and print validation results to ``stdout``: .. code-block:: text $ cat values.yaml | uw config validate --schema-file schema.jsonschema [2024-01-03T17:26:29] INFO 0 UW schema-validation errors found -* However, reading the schema from ``stdin`` is **not** supported: +* However, reading the *schema* from ``stdin`` is **not** supported: .. code-block:: text @@ -552,7 +558,7 @@ and YAML file ``values.yaml`` with content [2024-01-03T17:31:19] ERROR On instance['values']: [2024-01-03T17:31:19] ERROR {'greeting': 'Hello'} -* Request verbose log output: +* To request verbose log output: .. code-block:: text @@ -572,7 +578,7 @@ and YAML file ``values.yaml`` with content $ uw config validate --schema-file schema.jsonschema --input-file values.yaml --verbose 2>validate.log - The content of ``validate.log``: + The contents of ``validate.log``: .. code-block:: text diff --git a/docs/sections/user_guide/cli/mode_rocoto.rst b/docs/sections/user_guide/cli/mode_rocoto.rst index cfc21e6fa..ae3c97be2 100644 --- a/docs/sections/user_guide/cli/mode_rocoto.rst +++ b/docs/sections/user_guide/cli/mode_rocoto.rst @@ -54,7 +54,7 @@ More information about the structured UW YAML file for Rocoto can be found :any: Examples ^^^^^^^^ -The examples that follow use a UW YAML file ``rocoto.yaml`` with content +The examples in this section use a UW YAML file called ``rocoto.yaml`` with the following contents: .. code-block:: python @@ -82,7 +82,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content envars: person: siri -* Realize a UW YAML input file to ``stdout`` in Rocoto XML format: +* To realize a UW YAML input file to ``stdout`` in Rocoto XML format: .. code-block:: xml @@ -110,7 +110,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content -* Realize a UW YAML input file to a file named ``rocoto.xml``: +* To realize a UW YAML input file to a file named ``rocoto.xml``: .. code-block:: text @@ -143,7 +143,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content -* Read the UW YAML from ``stdin`` and write the XML to ``stdout``: +* To read the UW YAML from ``stdin`` and write the XML to ``stdout``: .. code-block:: xml @@ -171,7 +171,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content -* Realize a UW YAML input file to a file named ``rocoto.xml`` in quiet mode: +* To realize a UW YAML input file to a file named ``rocoto.xml`` in quiet mode: .. code-block:: text @@ -180,7 +180,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content The contents of ``rocoto.xml`` are unchanged from the previous example. -* For increased verbosity realizing a UW YAML file to a file named ``rocoto.xml``: +* To realize a UW YAML file to a file named ``rocoto.xml`` with verbose log output: .. note:: This output has been shortened for demonstration purposes. @@ -228,7 +228,7 @@ The examples that follow use a UW YAML file ``rocoto.yaml`` with content Examples ^^^^^^^^ -The examples that follow use a Rocoto XML file ``rocoto.xml`` with the following content: +The examples in this section use a Rocoto XML file called ``rocoto.xml`` with the following content: .. code-block:: xml :linenos: diff --git a/docs/sections/user_guide/cli/mode_template.rst b/docs/sections/user_guide/cli/mode_template.rst index bdecf9df3..acd0f84fe 100644 --- a/docs/sections/user_guide/cli/mode_template.rst +++ b/docs/sections/user_guide/cli/mode_template.rst @@ -63,20 +63,20 @@ The ``uw`` mode for handling :jinja2:`Jinja2 templates`. Examples ^^^^^^^^ -The examples that follow use template file ``template`` with content +The examples in this section use a template file named ``template`` with the following contents: .. code-block:: jinja {{ greeting }}, {{ recipient }}! -and YAML file ``values.yaml`` with content +and a YAML file called ``values.yaml`` with the following contents: .. code-block:: yaml greeting: Hello recipient: World -* Show the values needed to render the template: +* To show the values needed to render the template: .. code-block:: text @@ -85,16 +85,16 @@ and YAML file ``values.yaml`` with content [2023-12-18T19:16:08] INFO greeting [2023-12-18T19:16:08] INFO recipient -* Render the template to ``stdout``: +* To render the template to ``stdout``: .. code-block:: text $ uw template render --input-file template --values-file values.yaml Hello, World! - Shell redirection via ``|``, ``>``, et al may also be used to stream output to a file, another process, etc. + Shell redirection via ``|``, ``>``, et al. may also be used to stream output to a file, another process, etc. -* Render the template to a file via command-line argument: +* To render the template to a file via command-line argument: .. code-block:: text @@ -113,14 +113,14 @@ and YAML file ``values.yaml`` with content $ uw template render --input-file template --values-file values.yaml --dry-run [2023-12-18T19:38:15] INFO Hello, World! -* Read the template from ``stdin`` and render to ``stdout``: +* To read the template from ``stdin`` and render to ``stdout``: .. code-block:: text $ cat template | uw template render --values-file values.yaml Hello, World! -* If the values file has an unrecognized (or no) extension, ``uw`` will not know how to parse its content: +* If the values file has an unrecognized (or no) extension, ``uw`` will not know how to parse its contents: .. code-block:: text @@ -142,21 +142,21 @@ and YAML file ``values.yaml`` with content [2023-12-18T19:30:05] ERROR Required value(s) not provided: [2023-12-18T19:30:05] ERROR recipient - But values may be supplemented by ``key=value`` command-line arguments, e.g. + But values may be supplemented by ``key=value`` command-line arguments. For example: .. code-block:: text $ uw template render --input-file template --values-file values.yaml recipient=Reader Hello, Reader! - Such ``key=value`` arguments may also be used to *override* file-based values + Such ``key=value`` arguments may also be used to *override* file-based values: .. code-block:: text $ uw template render --input-file template --values-file values.yaml recipient=Reader greeting="Good day" Good day, Reader! -* Request verbose log output: +* To request verbose log output: .. code-block:: text @@ -206,7 +206,7 @@ and YAML file ``values.yaml`` with content [2023-12-18T23:27:04] DEBUG --------------------------------------------------------------------- [2023-12-18T23:27:04] DEBUG Read initial values from values.yaml -* Non-YAML-formatted files may also be used as values sources. For example, ``template`` +* Non-YAML-formatted files may also be used as value sources. For example, ``template`` .. code-block:: jinja @@ -228,7 +228,7 @@ and YAML file ``values.yaml`` with content $ uw template render --input-file template --values-file values.nml Hello, World! - Note that ``ini`` and ``nml`` configs are, by definition, depth-2 configs, while ``sh`` configs are depth-1 and ``yaml`` configs have arbitrary depth. + Note that ``ini`` and ``nml`` configs are, by definition, depth-2 configs, while ``sh`` configs are depth-1, and ``yaml`` configs have arbitrary depth. .. _cli_template_translate_examples: @@ -262,22 +262,22 @@ and YAML file ``values.yaml`` with content Examples ^^^^^^^^ -The examples that follow use atparse-formatted template file ``atparse.txt`` with content +The examples in this section use atparse-formatted template file ``atparse.txt`` with the following contents: .. code-block:: text @[greeting], @[recipient]! -* Convert an atparse-formatted template file to Jinja2 format: +* To convert an atparse-formatted template file to Jinja2 format: .. code-block:: text $ uw template translate --input-file atparse.txt {{greeting}}, {{recipient}}! - Shell redirection via ``|``, ``>``, et al may also be used to stream output to a file, another process, etc. + Shell redirection via ``|``, ``>``, et al. may also be used to stream output to a file, another process, etc. -* Convert the template to a file via command-line argument: +* To convert the template to a file via command-line argument: .. code-block:: text diff --git a/docs/sections/user_guide/installation.rst b/docs/sections/user_guide/installation.rst index 6b9ac3b1c..e8755759b 100644 --- a/docs/sections/user_guide/installation.rst +++ b/docs/sections/user_guide/installation.rst @@ -1,7 +1,9 @@ Installation ============ -**NOTE:** Developers should visit the :doc:`Developer Setup <../contributor_guide/developer_setup>` section located in the :doc:`Contributor Guide <../contributor_guide/index>`. +.. note:: + + Developers should visit the :doc:`Developer Setup <../contributor_guide/developer_setup>` section located in the :doc:`Contributor Guide <../contributor_guide/index>`. The recommended installation mechanism uses the Python package and virtual-environment manager :conda:`conda<>`. Specifically, these instructions assume use of the :miniforge:`Miniforge<>` variant of :miniconda:`Miniconda<>`, built to use, by default, packages from the :conda-forge:`conda-forge<>` project. Users of the original :miniconda:`Miniconda<>` or the :anaconda:`Anaconda distribution<>` should add the flags ``-c conda-forge --override-channels`` to ``conda`` commands to specify the required package channels. @@ -11,7 +13,7 @@ Use an Existing conda Installation Install Into an Existing Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To install ``uwtools`` into an existing environment in an existing conda (e.g. :miniforge:`Miniforge<>`, :miniconda:`Miniconda<>`, :anaconda:`Anaconda<>`) installation: +To install ``uwtools`` into an existing environment in an existing conda (e.g., :miniforge:`Miniforge<>`, :miniconda:`Miniconda<>`, :anaconda:`Anaconda<>`) installation: #. Activate that environment. #. Identify the ``uwtools`` version number to install from the available versions shown by ``conda search -c ufs-community --override-channels uwtools``. @@ -36,7 +38,7 @@ To create a standalone conda environment providing ``uwtools``: Use a Fresh Miniforge Installation ---------------------------------- -If no existing conda installation is available, install :miniforge:`Miniforge<>` +If no existing conda installation is available, install :miniforge:`Miniforge<>`. .. include:: ../../shared/miniforge3_instructions.rst @@ -62,10 +64,10 @@ Build the ``uwtools`` Package Locally .. code-block:: text - conda search -c local --override-channels uwtools # do not add -c conda-forge to this command + conda search -c local --override-channels uwtools # do not add -c conda-forge to this command #. Optionally, create an environment from the newly built package: .. code-block:: text - conda create -n uwtools -c local uwtools[=] # specify version if multiple choices are available + conda create -n uwtools -c local uwtools[=] # specify version if multiple choices are available diff --git a/docs/sections/user_guide/uw_yaml/rocoto_yaml.rst b/docs/sections/user_guide/uw_yaml/rocoto_yaml.rst index 28236c0fd..90a305f3a 100644 --- a/docs/sections/user_guide/uw_yaml/rocoto_yaml.rst +++ b/docs/sections/user_guide/uw_yaml/rocoto_yaml.rst @@ -33,7 +33,11 @@ Starting at the top level of the UW YAML config for Rocoto, there are several re UW YAML Keys ^^^^^^^^^^^^ -``attrs:``: This section allows users to define any number of the available attributes to the ```` tag in Rocoto's native XML language. Set each with a key/value that matches the exact syntax needed by Rocoto. These attributes work to fill in the ```` tag. For example, +.. note:: + + For additional information on Rocoto tags mentioned in this section (e.g., ````, ````, ````), please see the :rocoto:`Rocoto documentation<>`. + +``attrs:`` -- This section allows users to define any number of the available attributes to the ```` tag in Rocoto's native XML language. Set each with a key/value that matches the exact syntax needed by :rocoto:`Rocoto<>`. These attributes work to fill in the ```` tag. For example: .. code-block:: xml @@ -41,13 +45,13 @@ UW YAML Keys ... -``cycledef:``: This section is a list of grouped cycle definitions for a workflow. Any number of ``cycledef:`` keys is supported. Similar to ``attrs:`` for the ``workflow:`` level, this section has an ``attrs:`` key that follows the exact requirements of those in the Rocoto XML language. The ``spec:`` key is required and supports either the "start, stop, step" syntax, or the "crontab-like" method supported by Rocoto. The example above translates to a single ```` tag: +``cycledef:`` -- This section is a list of grouped cycle definitions for a workflow. Any number of ``cycledef:`` keys is supported. Similar to ``attrs:`` for the ``workflow:`` level, this section has an ``attrs:`` key that follows the exact requirements of those in the Rocoto XML language. The ``spec:`` key is required and supports either the "start, stop, step" syntax, or the "crontab-like" method supported by Rocoto. The example above translates to a single ```` tag: .. code-block:: xml 202209290000 202209300000 06:00:00 -``entities:``: This section defines key/value pairs -- each rendered as ```` -- to translate to named entities (variables) in XML. The example above would yield: +``entities:`` -- This section defines key/value pairs --- each rendered as ```` --- to translate to named entities (variables) in XML. The example above would yield: .. code-block:: xml @@ -57,13 +61,13 @@ UW YAML Keys ]> -``log:``: This is a path-like string that defines where to put the Rocoto logs. It corresponds to the ```` tag. For example: +``log:`` -- This is a path-like string that defines where to put the Rocoto logs. It corresponds to the ```` tag. For example: .. code-block:: xml /some/path/to/&FOO; -``tasks:``: This section is explained in the ``Tasks Section``. +``tasks:`` -- This section is explained in the `Tasks Section`_. Using Cycle Strings ^^^^^^^^^^^^^^^^^^^ @@ -86,7 +90,7 @@ In the example, the resulting log would appear in the XML file as: /some/path/to/&FOO; -The ``attrs:`` block is optional within the ``cyclestr:`` block, and can be used to specify the cycle offset. +The ``attrs:`` block is optional within the ``cyclestr:`` block and can be used to specify the cycle offset. Tasks Section ------------- @@ -120,13 +124,13 @@ Each task is named by its UW YAML key. Blocks under ``tasks:`` prefixed with ``t ... -where the ``attrs:`` section may set any of the Rocoto-allowed XML attributes. The ```` tag will, by default, use the same name, but may be overridden with an explicit ``jobname:`` key under the task. +where the ``attrs:`` section may set any of the Rocoto-allowed XML attributes. The ```` tag will, by default, use the same name but may be overridden with an explicit ``jobname:`` key under the task. The name of the task can be any string accepted by Rocoto as a task name (including additional underscores), but must contain the leading ``task_`` to be recognized as a task. -``command:``: The command that will be run in the batch job. +``command:`` -- The command that will be run in the batch job. -``envars:``: Any number of key/value pairs defining bash variable names and their corresponding values, to be exported to the environment in which ```` will run, each rendered in XML like this: +``envars:`` -- Any number of key/value pairs defining bash variable names and their corresponding values, to be exported to the environment in which ```` will run, each rendered in XML like this: .. code-block:: xml @@ -135,7 +139,7 @@ The name of the task can be any string accepted by Rocoto as a task name (includ siri -``dependencies:``: [Optional] Any number of dependencies accepted by Rocoto. This section is described in more detail below. +``dependencies:`` -- [Optional] Any number of dependencies accepted by Rocoto. This section is described in more detail below. The other keys not specifically mentioned here follow the same conventions as described in the :rocoto:`Rocoto<>` documentation. @@ -149,7 +153,7 @@ UW YAML dependency key names should mirror Rocoto XML dependency tag names, opti Specifying Tag Attributes ^^^^^^^^^^^^^^^^^^^^^^^^^ -Each of the dependencies that require attributes (the ``key="value"`` parts inside the XML tag) can be specified with an ``attrs:`` block. For example: +Each of the dependencies that requires attributes (the ``key="value"`` parts inside the XML tag) can be specified with an ``attrs:`` block. For example: .. code-block:: yaml @@ -210,12 +214,12 @@ This would result in Rocoto XML in this form: The ``datadep_foo:`` and ``datadep_bar:`` UW YAML keys were named arbitrarily after the first ``_``, but could have been even more descriptive such as ``datadep_foo_file:`` or ``datadep_foo_text:``. The important part is that the YAML key prefix matches the Rocoto XML tag name. -This example also demonstrates the use of Rocoto's **boolean operator tags** in the structured UW YAML, e.g. ````, ````, etc.. The structure follows the tree in the Rocoto XML language in that each of the sub-elements of the ```` tag translate to sub-trees in UW YAML. Multiple of the boolean operator tags can be set at the same level just as with any other tag type by adding a descriptive suffix starting with an underscore. In the above example, the ``and:`` key could have equivalently been named ``and_data_files:`` to achieve an identical Rocoto XML result. +This example also demonstrates the use of Rocoto's **boolean operator tags** in the structured UW YAML, e.g., ````, ````, etc. The structure follows the tree in the Rocoto XML language in that each of the subelements of the ```` tag translates to a subtree in UW YAML. Multiple boolean operator tags can be set at the same level, just as with any other tag type, by adding a descriptive suffix starting with an underscore. In the above example, the ``and:`` key could have equivalently been named ``and_data_files:`` to achieve an identical Rocoto XML result. Defining Metatasks ------------------ -A Rocoto ``metatask`` expands into one or more tasks via substitution of values, defined under the ``var:`` key, into placeholders bracketed with pound signs. Each variable must provide the same number of values. Here is UW YAML that localizes a greeting to a variety of languages: +A Rocoto ``metatask`` expands into one or more tasks (defined under the ``var:`` key) via substitution of values. Placeholders bracketed with pound signs are replaced with the values included under the ``var:`` key. Each variable must provide the same number of values. Here is UW YAML that localizes a greeting to a variety of languages: .. Leaving this one as text because the #'s show up as comments, which is unhelpful @@ -246,6 +250,8 @@ This translates to Rocoto XML (whitespace added for readability): +This example of Rocoto XML will be expanded during the workflow's execution to generate three individual tasks: ``hello``, ``hola``, and ``bonjour``. + UW YAML Definitions ------------------- @@ -337,7 +343,7 @@ Defining the Workflow Log /some/path/to/workflow.log -A cycle string may be specified here, instead. +A cycle string may be specified here instead. .. code-block:: yaml @@ -451,7 +457,7 @@ Boolean operator keys operate on **one or more additional dependency entries** f -Comparison Depenedencies +Comparison Dependencies ^^^^^^^^^^^^^^^^^^^^^^^^ The ``streq:`` and ``strneq:`` keys compare the values in their ``left:`` and ``right:`` children, and accept ``cyclestr:`` blocks as well as simple strings. @@ -537,7 +543,7 @@ Dependency Keys /path/to/a/file.txt - * The ``value:`` key accepts a ``cyclestr:`` block. + * The ``value:`` key accepts a ``cyclestr:`` block. * The ``timedep:`` key defines a dependency on a real-world time: @@ -569,7 +575,7 @@ Dependency Keys * The ``command:`` key accepts a ``cyclestr:`` block. - * The ``sh:`` key may be suffixed with an underscore and a name to provide a unique name for the dependency, e.g. ``sh_count_grib:`` would translate to XML tag ````. + * The ``sh:`` key may be suffixed with an underscore and a name to provide a unique name for the dependency, e.g., ``sh_count_grib:`` would translate to XML tag ````. * The optional attributes ``runopt`` and ``shell`` are accepted under an ``attrs:`` key. See the :rocoto:`Rocoto documentation<>` for details. The ``metatask:`` Key @@ -613,7 +619,7 @@ This will run tasks named: graphics_mem002_v graphics_mem003_v -The XML will look like this +The XML will look like this: .. code-block:: xml diff --git a/docs/shared/miniforge3_instructions.rst b/docs/shared/miniforge3_instructions.rst index 39c8823f4..9dec574f4 100644 --- a/docs/shared/miniforge3_instructions.rst +++ b/docs/shared/miniforge3_instructions.rst @@ -1,4 +1,4 @@ -This recipe uses the ``aarch64`` (64-bit ARM) Miniforge for Linux, and installs into ``$HOME/conda``. Adjust as necessary for the target system. +This recipe uses the ``aarch64`` (64-bit ARM) Miniforge for Linux and installs into ``$HOME/conda``. Adjust as necessary for the target system. #. Download, install, and activate the latest :miniforge3:`Miniforge<>` for the target system. @@ -10,7 +10,7 @@ This recipe uses the ``aarch64`` (64-bit ARM) Miniforge for Linux, and installs source ~/conda/etc/profile.d/conda.sh conda activate - After initial installation, this conda may be activated with the command + After initial installation, this conda may be activated with the command: .. code-block:: text