Omnibus 2024-11-27

[maddenp-noaa]

Synopsis

• Update .github such that the Test workflow will be run for PRs or pushes to all branches. This means that tests will run for updates on release branches, which seems like it would have been a good idea to always be doing.
• Add setuptools to conda build requirements, as it appears not to be automatically installed with Python 3.13 now in CI jobs, but is required for the uwtools build.
• Some updated documentation about the !include tag.
• Some trivial doc changes per make docs.
• Fix a bug whereby Jinja2 expressions referencing tagged YAML values would render to strings including the tag as part of the string. The fix involves detecting tagged values and raising an exception to delay rendering the value until a later iteration of the dereferencing process, when the tagged value will have been converted to its simple type (int, bool, etc.). See below (and the new unit tests) for examples.

Given config.yaml

a: 2
b: 7
foo: !int "{{ a * b }}"
bar: !int "{{ foo }}"

Old behavior:

$ uw config realize -i config.yaml --output-format yaml
a: 2
b: 7
foo: 14
bar: !int '!int 14'

New behavior:

$ uw config realize -i config.yaml --output-format yaml
a: 2
b: 7
foo: 14
bar: 14

Type

• Bug fix (corrects a known issue)
• Documentation
• Tooling (CI, code-quality, packaging, revision-control, etc.)

Impact

• This is a non-breaking change (existing functionality continues to work as expected)

Checklist

• I have added myself and any co-authors to the PR's Assignees list.
• I have reviewed the documentation and have made any updates necessitated by this change.

[maddenp-noaa]

Show the keypath using dot notation, for consistency with other places where uwtools shows keypaths.

The rest of the changes in this function are cosmetic.

[maddenp-noaa]

If the rendered value parses to a UWYAMLConvert object (i.e. is prefixed with one of our !bool, !int, etc. tags), raise an exception to keep the original value intact for now; it may be possible to render it later after the value it references has itself been rendered and converted to a basic (bool, int, etc.) value.

[maddenp-noaa]

To keep the new tests -- see below -- DRY.

[maddenp-noaa]

The basic loader, handling tags !bool, !datetime, !float, and !int, has been factored out to uwtools.config.support so that it can be used by code outside YAMLConfig. This method obtains that loader, then adds !include support. It seemed far from straightforward to factor !include support out of YAMLConfig, and I think there are open questions about how the !include tag would interact with other features (e.g. an !include-tagged value that loads values tagged with other tags like !int et al, how these interact with Jinja2 value rendering, etc.) I think we'll have to cross that bridge when we come to it.

[maddenp-noaa]

I did a basic test involving tagged values and !include and things seem to work as I'd expect:

numbers.yaml

e_plus_half_pi: !float '{{ values.e + half_pi }}'
half_pi: !float '{{ values.pi / 2 }}'
values: !include [e.yaml, pi.yaml]

e.yaml

e: 2.718

pi.yaml

pi: 3.141

And

$ uw config realize --input-file numbers.yaml --output-format yaml
e_plus_half_pi: 4.2885
half_pi: 1.5705
values:
  e: 2.718
  pi: 3.141

[maddenp-noaa]

Here's the crux of the bugfix: If the rendered value turns out to be something (e.g. !int '{{ foo }}') that loads as a UWYAMLConvert object, reject the rendered value, and hold the old value, by raising an exception. The value may be rendered by a future iteration of the dereferencing loop when the value it refers to has been reduced to a basic value (a bool, int, etc.).

[maddenp-noaa]

Here's the loader that handles !bool, !int, etc. UW YAML tags.