From b99c3981466dcbec0e2126accfe687d9a855c2ab Mon Sep 17 00:00:00 2001 From: woutdenolf Date: Mon, 10 Oct 2022 20:09:51 +0200 Subject: [PATCH] NXdata: improve documentation regarding axes --- applications/NXcanSAS.nxdl.xml | 2 +- base_classes/NXdata.nxdl.xml | 351 +++++++++++++++----------------- base_classes/NXlog.nxdl.xml | 4 +- base_classes/NXmonitor.nxdl.xml | 2 +- 4 files changed, 163 insertions(+), 196 deletions(-) diff --git a/applications/NXcanSAS.nxdl.xml b/applications/NXcanSAS.nxdl.xml index e562257c4d..6ff164f6ad 100644 --- a/applications/NXcanSAS.nxdl.xml +++ b/applications/NXcanSAS.nxdl.xml @@ -1215,7 +1215,7 @@ - the wavelengths field (as a dimension scale) corresponding to this transmission + the wavelengths field (as coordinates) corresponding to this transmission diff --git a/base_classes/NXdata.nxdl.xml b/base_classes/NXdata.nxdl.xml index 774c653253..d750915500 100644 --- a/base_classes/NXdata.nxdl.xml +++ b/base_classes/NXdata.nxdl.xml @@ -42,36 +42,21 @@ These symbols will be used below to coordinate fields with the same shape. - rank of the ``DATA`` field - length of the ``AXISNAME`` field + rank of the ``DATA`` field(s) length of the ``x`` field length of the ``y`` field length of the ``z`` field - - - .. index:: plotting - - Array of strings holding the :ref:`names <validItemName>` of additional - signals to be plotted with the default :ref:`signal </NXdata@signal-attribute>`. - These fields or links *must* exist and be direct children of this NXdata group. - - Each auxiliary signal needs to be of the same shape as the default signal. - - .. NIAC2018: - https://www.nexusformat.org/NIAC2018Minutes.html - - .. index:: find the default plottable data .. index:: plotting .. index:: signal attribute value - Declares which NeXus field is the default. - The value is the :ref:`name <validItemName>` of the data field to be plotted. - This field or link *must* exist and be a direct child of this NXdata group. + The value is the :ref:`name <validItemName>` of the signal that contains + the default plottable data. This field or link *must* exist and be a direct child + of this NXdata group. It is recommended (as of NIAC2014) to use this attribute rather than adding a signal attribute to the field. @@ -79,40 +64,54 @@ for a summary of the discussion. + + + .. index:: plotting + + Array of strings holding the :ref:`names <validItemName>` of additional + signals to be plotted with the :ref:`default signal </NXdata@signal-attribute>`. + These fields or links *must* exist and be direct children of this NXdata group. + + Each auxiliary signal needs to be of the same shape as the default signal. + + .. NIAC2018: + https://www.nexusformat.org/NIAC2018Minutes.html + + .. index:: plotting - Array of strings holding the :ref:`names <validItemName>` of - the independent data fields used in the default plot for all of - the dimensions of the :ref:`signal </NXdata@signal-attribute>` - as well as any :ref:`auxiliary signals </NXdata@auxiliary_signals-attribute>`. + The coordinates of all elements in the :ref:`signal </NXdata@signal-attribute>` field + as well as any :ref:`auxiliary signal </NXdata@auxiliary_signals-attribute>` field + can be provided by the *axes* fields. - One name is provided for every dimension in the *signal* or *auxiliary signal* fields. + The :ref:`names <validItemName>` of these *axes* fields are provides as an array of strings + in the *axes* attribute of this NXdata group. The axes fields *must* exist and be direct children of this NXdata group. - The *axes* values are the names of fields or links that *must* exist and be direct - children of this NXdata group. - - An axis slice is specified using a field named ``AXISNAME_indices`` - as described below (where the text shown here as ``AXISNAME`` is to be - replaced by the actual field name). + One *axis* field can span multiple signal dimensions. In that case the attribute + :ref:`AXISNAME_indices </NXdata@AXISNAME_indices-attribute>` associated to the :ref:`AXISNAME field </NXdata/AXISNAME-field>` + is needed to describe which dimensions. The shape (rank and size of each dimension) of the :ref:`AXISNAME field </NXdata/AXISNAME-field>` + must be equal to the shape of the signal dimensions it spans. + + The total rank of all axes must be smaller or equal then the rank of the :ref:`DATA fields </NXdata/DATA-field>` (``dataRank``) + and the number of strings in the ``axes`` attribute must be equal to this rank as well. Use "." as axis name when the data does not have + coordinates along a dimension. - When no default axis is available for a particular dimension - of the plottable data, use a "." in that position. - Such as:: + For example:: @axes=["time", ".", "."] - Since there are three items in the list, the *signal* field - must be a three-dimensional array (rank=3). The first dimension - is described by the values of a one-dimensional array named ``time`` - while the other two dimensions have no fields to be used as dimension scales. + indicates that the :ref:`signal </NXdata/DATA-field>` must be a three-dimensional array (rank=3). The axis field with the name "time" + must be a one-dimensional array (rank=1). The last two data dimensions do not have coordinates. - See examples provided on the NeXus wiki: + More examples are provided on the NeXus wiki: https://www.nexusformat.org/2014_axes_and_uncertainties.html - If there are no axes at all (such as with a stack of images), - the axes attribute can be omitted. + If there are no coordinates at all (such as with a stack of images), the axes attribute can be omitted. + + .. note:: When ``axes`` contains multiple strings, it must be saved as an actual array + of strings and not a single comma separated string. @@ -122,171 +121,131 @@ --> - Each ``AXISNAME_indices`` attribute indicates the dependency - relationship of the ``AXISNAME`` field (where ``AXISNAME`` - is the name of a field that exists in this ``NXdata`` group) - with one or more dimensions of the plottable data. - - Integer array that defines the indices of the *signal* field - (that field will be a multidimensional array) - which need to be used in the *AXISNAME* field in - order to reference the corresponding axis value. + The ``AXISNAME_indices`` attribute is associated to the :ref:`AXISNAME field </NXdata/AXISNAME-field>`. + The :ref:`name <validItemName>` ``AXISNAME`` is also used in the :ref:`axes attribute </NXdata@axes-attribute>`. + + The ``AXISNAME_indices`` attribute is a single integer or an array of integers that defines which :ref:`data </NXdata/DATA-field>` + dimensions are spanned by the associated axis. The first dimension index is ``0`` (zero). + + An example with axes that span one dimension each:: - The first index of an array is ``0`` (zero). + data_2d:NXdata + @signal="data" + @axes=["time", "pressure"] + @time_indices=0 + @pressure_indices=1 + data: float[1000,20] + time: float[1000] + pressure: float[20] - Here, *AXISNAME* is to be replaced by the name of each - field described in the ``axes`` attribute. - An example with 2-D data, :math:`d(t,P)`, will illustrate:: + An example with axes that span one dimension ("time") and two dimensions ("pressure"):: - data_2d:NXdata - @signal="data" - @axes=["time", "pressure"] - @time_indices=0 - @pressure_indices=1 - data: float[1000,20] - time: float[1000] - pressure: float[20] + data_2d:NXdata + @signal="data" + @axes=["pressure", "time", "pressure"] + @time_indices=1 + @pressure_indices=[2, 0] + data: float[20,1000,30] + time: float[1000] + pressure: float[30,20] - This attribute is to be provided in all situations. - However, if the indices attributes are missing - (such as for data files written before this specification), - file readers are encouraged to make their best efforts - to plot the data. - Thus the implementation of the - ``AXISNAME_indices`` attribute is based on the model of - "strict writer, liberal reader". + Note that eventhough *data* is two-dimensional in the first example and three-dimensional + in the second example, we only have two coordinates so the coordinate space is two-dimensional + in both cases (*pressure* and *time*). - .. note:: Attributes potentially containing multiple values - (axes and _indices) are to be written as string or integer arrays, - to avoid string parsing in reading applications. + In this example, coordinate space is three-dimensional (*pressure*, *time* and a third unspecified + dimension):: + + data_2d:NXdata + @signal="data" + @axes=["pressure", "time", "."] + @time_indices=1 + @pressure_indices=0 + data: float[20,1000,30] + time: float[1000] + pressure: float[20] + + Finally you can have multiple coordinates for the same dimension. For example take + the same three-dimensional coordinate space as the previous example, but this time the + first dimension has two alternative coordinates (*pressure* being the default). So we + have *pressure*/*temperature*, *time* and a third unspecified dimension:: + + data_2d:NXdata + @signal="data" + @axes=["pressure", "time", "."] + @time_indices=1 + @pressure_indices=0 + @temperature_indices=0 + data: float[20,1000,30] + time: float[1000] + pressure: float[20] + temperature: float[20] + + The ``AXISNAME_indices`` attributes are to be provided if the indices cannot be unambiguously derived + from the names in the :ref:`axes attribute </NXdata@axes-attribute>`. + + .. note:: When ``AXISNAME_indices`` contains multiple integers, it must be saved as an actual array + of integers and not a comma separated string. - :ref:`NXdata` describes the plottable data and related dimension scales. + :ref:`NXdata` is used to implement one of the basic motivations of NeXus: to provide a default plot associated + to a NeXus group such as an :ref:`NXentry` group. :ref:`NXdata` describes plottable data (also referred to as + signals or dependent variables) and associated coordinates (also referred to as axes or independent variables). .. index:: plotting - It is strongly recommended that there is at least one :ref:`NXdata` - group in each :ref:`NXentry` group. - Note that the fields named ``AXISNAME`` and ``DATA`` - can be defined with different names. - (Upper case is used to indicate that the actual name is left to the user.) - The ``signal`` and ``axes`` attributes of the - ``data`` group define which items - are plottable data and which are *dimension scales*, respectively. + The :ref:`DATA fields </NXdata/DATA-field>` contain the signal values to be plotted. The name of the field + to be used as the *default plot signal* is provided by the :ref:`signal attribute </NXdata@signal-attribute>`. + The names of the fields to be used a *secondary plot signals* are provided by the :ref:`auxiliary_signals attribute</NXdata@auxiliary_signals-attribute>`. - :ref:`NXdata` is used to implement one of the basic motivations in NeXus, - to provide a default plot for the data of this :ref:`NXentry`. The actual data - might be stored in another group and (hard) linked to the :ref:`NXdata` group. - - * Each :ref:`NXdata` group will define one field as the default - plottable data. The value of the ``signal`` attribute names this field. - Additional fields may be used to describe the dimension scales and - uncertainities. - The ``auxiliary_signals`` attribute is a list of the other fields - to be plotted with the ``signal`` data. - * The plottable data may be of arbitrary rank up to a maximum - of ``NX_MAXRANK=32`` (for compatibility with backend file formats). - * The plottable data will be named as the value of - the group ``signal`` attribute, such as:: - - data:NXdata - @signal = "counts" - @axes = "mr" - @mr_indices = 0 - counts: float[100] --> the default dependent data - mr: float[100] --> the default independent data - - The field named in the ``signal`` attribute **must** exist, either - directly as a NeXus field or defined through a link. + .. index:: axes (attribute) + .. index:: coordinates - * The group ``axes`` attribute will name the - *dimension scale* associated with the plottable data. - - If available, the standard deviations of the data are to be - stored in a data set of the same rank and dimensions, with the name ``errors``. - - * For each data dimension, there should be a one-dimensional array - of the same length. - * These one-dimensional arrays are the *dimension scales* of the - data, *i.e*. the values of the independent variables at which the data - is measured, such as scattering angle or energy transfer. + The :ref:`AXISNAME fields </NXdata/AXISNAME-field>` contain the coordinates associated to the data values. + The names of the fields to be used as coordinates are provided by the :ref:`axes attribute </NXdata@axes-attribute>`. + One axis field provides the coordinates along one or more data dimensions. - .. index:: link - .. index:: axes (attribute) + The fields :ref:`DATA </NXdata/DATA-field>` and :ref:`AXISNAME </NXdata/AXISNAME-field>` + should be defined with different names. (Upper case is used to indicate that the actual name is left to the user.) - The preferred method to associate each data dimension with - its respective dimension scale is to specify the field name - of each dimension scale in the group ``axes`` attribute as a string list. - Here is an example for a 2-D data set *data* plotted - against *time*, and *pressure*. (An additional *temperature* data set - is provided and could be selected as an alternate for the *pressure* axis.):: - - data_2d:NXdata - @signal="data" - @axes=["time", "pressure"] - @pressure_indices=1 - @temperature_indices=1 - @time_indices=0 - data: float[1000,20] - pressure: float[20] - temperature: float[20] - time: float[1000] - - .. rubric:: Old methods to identify the plottable data + Standard deviations on data values as well as coordinates can be provided by :ref:`FIELDNAME_errors fields </NXdata/FIELDNAME_errors-field>` + where ``FIELDNAME`` is the name of a :ref:`DATA field </NXdata/DATA-field>` or an :ref:`AXISNAME field </NXdata/AXISNAME-field>`. - There are two older methods of associating - each data dimension to its respective dimension scale. - Both are now out of date and - should not be used when writing new data files. - However, client software should expect to see data files - written with any of these methods. + An example for three-dimensional data with coordinates along the first two dimensions:: - * One method uses the ``axes`` - attribute to specify the names of each *dimension scale*. + data:NXdata + @signal = "counts1" + @auxiliary_signals = ["counts2", "counts3", "counts4"] + @axes = ["x", "y", "."] + @x_indices = 0 + @y_indices = 1 + counts1: float[10, 20, 100] --> the default signal + counts2: float[10, 20, 100] + counts3: float[10, 20, 100] + counts4: float[10, 20, 100] + x: float[10] --> coordinates along the first dimension + x@long_name: "Horizontal" + x@units: "um" + y: float[20] --> coordinates along the second dimension + y@units: "um" + y@long_name: "Vertical" - * The oldest method uses the ``axis`` attribute on each - *dimension scale* to identify - with an integer the axis whose value is the number of the dimension. - - .. index: !plot; axis label - plot, axis units - units - dimension scale - - Each axis of the plot may be labeled with information from the - dimension scale for that axis. The optional ``@long_name`` attribute - is provided as the axis label default. If ``@long_name`` is not - defined, then use the name of the dimension scale. A ``@units`` attribute, - if available, may be added to the axis label for further description. - See the section :ref:`Design-Units` for more information. - - .. index: !plot; axis title - - The optional ``title`` field, if available, provides a suggested - title for the plot. If no ``title`` field is found in the :ref:`NXdata` - group, look for a ``title`` field in the parent :ref:`NXentry` group, - with a fallback to displaying the path to the :ref:`NXdata` group. - - NeXus is about how to find and annotate the data to be plotted - but not to describe how the data is to be plotted. - (https://www.nexusformat.org/NIAC2018Minutes.html#nxdata-plottype--attribute) + .. note:: ``NXdata`` provides data and coordinates to be plotted but + does not describe how the data is to be plotted. + https://www.nexusformat.org/NIAC2018Minutes.html#nxdata-plottype--attribute - Dimension scale defining an axis of the data. - Client is responsible for defining the dimensions of the data. - The name of this field may be changed to fit the circumstances. - Standard NeXus client tools will use the attributes to determine - how to use this field. - - - - A *dimension scale* must have a rank of 1 and has length ``n``. - - - + Coordinate values along one or more data dimensions. + Client is responsible for defining the dimensions of the data. + The total rank of all ``AXISNAME`` listed in :ref:`axes attribute </NXdata@axes-attribute>` + must be equal to the rank of the :ref:`DATA fields </NXdata/DATA-field>` (rank=``dataRank``). + The name of this field may be changed to fit the circumstances. + Standard NeXus client tools will use the attributes to determine + how to use this field. + Axis label @@ -327,12 +286,13 @@ The name of this field may be changed to fit the circumstances. Standard NeXus client tools will use the attributes to determine how to use this field. + + The maximum rank is ``32`` for compatibility with backend file formats. The rank (``dataRank``) of the ``data`` must satisfy ``1 <= dataRank <= NX_MAXRANK=32``. - At least one ``dim`` must have length ``n``. - Defines the names of the dimension scales + Defines the names of the coordinates (independent axes) for this data set as a colon-delimited array. NOTE: The ``axes`` attribute is the preferred @@ -370,10 +330,8 @@ - The ``errors`` must have - the same rank (``dataRank``) - as the ``data``. - At least one ``dim`` must have length "n". + The ``errors`` must have the same rank (``dataRank``) + as the ``data``. @@ -399,7 +357,10 @@ This is an array holding the values to use for the x-axis of - data. The units must be appropriate for the measurement. + data. The units must be appropriate for the measurement. + + This is a special case of a :ref:`AXISNAME field </NXdata/AXISNAME-field>` + kept for backward compatiblity. @@ -408,7 +369,10 @@ This is an array holding the values to use for the y-axis of - data. The units must be appropriate for the measurement. + data. The units must be appropriate for the measurement. + + This is a special case of a :ref:`AXISNAME field </NXdata/AXISNAME-field>` + kept for backward compatiblity. @@ -417,7 +381,10 @@ This is an array holding the values to use for the z-axis of - data. The units must be appropriate for the measurement. + data. The units must be appropriate for the measurement. + + This is a special case of a :ref:`AXISNAME field </NXdata/AXISNAME-field>` + kept for backward compatiblity. diff --git a/base_classes/NXlog.nxdl.xml b/base_classes/NXlog.nxdl.xml index 813d0e918d..2a71845650 100644 --- a/base_classes/NXlog.nxdl.xml +++ b/base_classes/NXlog.nxdl.xml @@ -49,8 +49,8 @@ can be used to accomodate non standard clocks. - This method of storing logged data helps to distinguish - instances in which a variable is a dimension scale of the data, in which case it is stored + This method of storing logged data helps to distinguish instances in which a variable contains signal or + coordinate values of plottable data, in which case it is stored in an :ref:`NXdata` group, and instances in which it is logged during the run, when it should be stored in an :ref:`NXlog` group. diff --git a/base_classes/NXmonitor.nxdl.xml b/base_classes/NXmonitor.nxdl.xml index fce17418f2..e6444a28e9 100644 --- a/base_classes/NXmonitor.nxdl.xml +++ b/base_classes/NXmonitor.nxdl.xml @@ -30,7 +30,7 @@ A monitor of incident beam data. It is similar to the :ref:`NXdata` groups containing - monitor data and its associated dimension scale, e.g. time_of_flight or + monitor data and its associated coordinates, e.g. time_of_flight or wavelength in pulsed neutron instruments. However, it may also include integrals, or scalar monitor counts, which are often used in both in both pulsed and steady-state instrumentation.