NXdata: improve documentation regarding axes

[woutdenolf]

Closes #1212

In this PR we refactor the NXdata definition to resolve inconsistencies in definition of @axes, AXISNAME_indices and AXISNAME. NIAC discussions:

• Telco_20230314: illustrate the problem with the current definition 2023_03_niac.pdf
• Telco_20230419: provide answers to NXdata - @AXISNAME_indices confusion #1212 (comment) and NXdata - @AXISNAME_indices confusion #1212 (comment)

The proposed changes do not change anything to the current NXdata, they just clear up ambiguity:

• @AXISNAME_indices: data dimension(s) spanned by AXISNAME. When missing it defaults to the index or indices of AXISNAME in @axes.
• @AXISNAME_indices: a single integer or an array of integers.
• the rank of AXISNAME must be equal to the number of dimensions it spans
• dimension scales is a confusing term which comes from HDF5 terminology: use axes, axis coordinates or independent variables depending on the context
• move all complex explanations from the fields/attribute docs to the global NXdata description
• divide the global NXdata description in sub-sections: signals, axes, uncertainties and examples
• add better examples

Proper code examples with be added in a separate merge request: #1253

A python reader would have to do something like this to figure out what all the axes are and to which data dimensions they belong

from typing import Dict, List, Sequence
import h5py

def get_axes_dims(nxdata: h5py.Group) -> Dict[str,List[int]]:
    axes_data_dims = dict()
    axes = nxdata.attrs.get("axes", list())
    if isinstance(axes, str):
        axes = [axes]
    for axisname in set(axes):
        if axisname == ".":
            continue
        data_dims = nxdata.attrs.get(f"{axisname}_indices", None)
        if data_dims is None:
            data_dims = [i for i,s in enumerate(axes) if s == axisname]
        elif not isinstance(data_dims, Sequence):
            data_dims = [data_dims]
        else:
            data_dims = list(data_dims)
        axes_data_dims[axisname] = data_dims
    return axes_data_dims

Rendering of the PR

https://hdf5.gitlab-pages.esrf.fr/nexus/nxdata_axes/classes/base_classes/NXdata.html

[woutdenolf]

Changes:

• added the simple x-y example
• applied suggestion NXdata: improve documentation regarding axes #1213 (comment)
• change the axes section to take into account NXdata: improve documentation regarding axes #1213 (comment) but with more emphasis on the rigourous logic of assigned axes to dimensions, i.e. "first look for AXISNAME_indices, if missing take the position(s) in @axes"

[woutdenolf]

For reference, I took @rayosborn's description of axes #1213 (comment)

Axes:

The AXISNAME fields contain the coordinates associated with the data values. AXISNAME fields are typically one-dimensional arrays, which annotate one of the dimensions. However, the fields can also have a rank greater than 1, in which case the rank of each AXISNAME must be equal to the number of dimensions it spans.

The names of the AXISNAME fields to be used as axis coordinates are listed in the axes attribute. When all the AXISNAME fields are one-dimensional, their positions in the axes attribute correspond to the data dimension being annotated. If one of the data dimensions has no AXISNAME field, the string “.” can be used in the corresponding index of the axes list.

When the rank of any of the AXISNAME fields is greater than 1, the AXISNAME_indices attributes is used to define the data dimension(s) spanned by the corresponding AXISNAME fields. It is strongly discouraged to define the AXISNAME_indices attribute for some AXISNAME fields but not for others."

and merged it with my original wording to obtain this

Axes:

The AXISNAME fields contain the axis coordinates associated with the data values. The names of all AXISNAME fields are listed in the axes attribute.

Rank

AXISNAME fields are typically one-dimensional arrays, which annotate one of the dimensions. However, the fields can also have a rank greater than 1, in which case the rank of each AXISNAME must be equal to the number of data dimensions it spans.

Dimensions

The data dimensions annotated by an AXISNAME field are defined by the AXISNAME_indices attribute. When this attribute is missing, the position(s) of the AXISNAME string in the axes attribute are used.

When all AXISNAME fields are one-dimensional, and none of the data dimensions have more than one axis, the AXISNAME_indices attributes are often omitted. If one of the data dimensions has no AXISNAME field, the string “.” can be used in the corresponding index of the axes list.

When providing AXISNAME_indices attributes it is recommended to do it for all axes.

I mainly had an issue with this

When all the AXISNAME fields are one-dimensional, their positions in the axes attribute correspond to the data dimension being annotated.
...
When the rank of any of the AXISNAME fields is greater than 1, the AXISNAME_indices attributes is used to define the data dimension(s) spanned by the corresponding AXISNAME fields.

This seems to imply the AXISNAME_indices are to be ignored when you only have rank 1 axes. This is logically very hard to implement mostly for HDF5 readers. In the merged version this becomes

The data dimensions annotated by an AXISNAME field are defined by the AXISNAME_indices attribute. When this attribute is missing, the position(s) of the AXISNAME string in the axes attribute are used.

This is logically much simpler. @rayosborn's description in spirit wants to say that most of our NXdata groups do not has AXISNAME_indices. So I added this paragraph

When all AXISNAME fields are one-dimensional, and none of the data dimensions have more than one axis, the AXISNAME_indices attributes are often omitted. If one of the data dimensions has no AXISNAME field, the string “.” can be used in the corresponding index of the axes list.

[woutdenolf]

To accommodate @phyy-nx who prefer positive wording I replaced

It is strongly discouraged to define the AXISNAME_indices attribute for some AXISNAME fields but not for others.

with

When providing AXISNAME_indices attributes it is recommended to do it for all axes.

[woutdenolf]

Split the Axes section into "simple" and "advanced" use cases

I did the splitting in another way with subsections rank and dimensions. The axes section should be easier to read now.

[phyy-nx]

Code review:

• Example 1, switch x and y
• Example 2, add note to caption that this is including AXISNAME_indices as an example
• Merge examples up into the docstring body.

[woutdenolf]

Applied the requested modifications from #1213 (comment): examples are merged with the doc sections and adapted to match those sections.

[phyy-nx]

From telco, let's get the same subcommittee mentioned above to re-review and approve the PR (@phyy-nx @woutdenolf @rayosborn @sanbrock). Further reviews are welcome. Target is November 2nd. Thanks!

[benajamin]

Looks like a big improvement to me!

[sanbrock]

LGTM

[phyy-nx]

Thanks! Looks good!

[rayosborn]

Great job.

[phyy-nx]

Ok! Based on Telcos and these approvals, @woutdenolf feel free to merge at your convenience. Thanks!

[woutdenolf]

Thanks for the review everyone!