Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better rewrite of NXdata scaling_factor and offset fields #1343

Merged
merged 16 commits into from
Sep 5, 2024

Conversation

phyy-nx
Copy link
Contributor

@phyy-nx phyy-nx commented Feb 6, 2024

Adds FIELDNAME_scaling_factor and FIELDNAME_offset as fields to NXdata. These fields should be used instead of scaling_factor and offset, as it is ambiguous which fields to apply them to in the case of multiple signals. _scaling_factor and _offset are added as reserved suffixes.

Additionally adds a formula for how to apply these fields:

corrected values = (FIELDNAME + offset) * scaling_factor

For NXmx, adds clarification these fields can be used as pedestal and gain correction fields, and defines these terms. Also the possible rank options are elaborated. These rank options were implied (in my opinion) in the original wording, but in NXmx I made it more explicit.

Closes #1332. Same set of changes as in #1333 with an additional commit to resolve feedback from #1333.

Change NXdata scaling_factor to refer to "plotted" data
Change NXdata to refer to "corrected" data, in addition to "physical" data, since it describes units of photons
This resolves ambiguity if there is more than one signal

For NXmx specify data_scaling_factor and data_offset since the field data is named in the NXdata group
- NXmx: fully define pedestal, bias, and gain and how they relate to offset and scaling_factor
- Add reserved suffixes _scaling_factor and _offset
- NXdata: define better how the stored values in FIELDNAME are converted to physical values
- NXdata: more clarification on how scaling_factor and offset without FIELDNAME are ambiguous
@phyy-nx
Copy link
Contributor Author

phyy-nx commented Feb 6, 2024

691f56a addresses comments from the review of #1333:

  • NXmx: fully define pedestal, bias, and gain and how they relate to offset and scaling_factor. Comments from @prjemian
  • Add reserved suffixes _scaling_factor and _offset. Comment from @PeterC-DLS
  • NXdata: define better how the stored values in FIELDNAME are converted to physical values. Comment from @benajamin
  • NXdata: more clarification on how scaling_factor and offset without FIELDNAME are ambiguous. Comment from @paulmillar

Thanks for the feedback! This is ready for further review :)

@phyy-nx
Copy link
Contributor Author

phyy-nx commented Jul 16, 2024

@paulmillar @PeterC-DLS I've come back to this after a while and responded to your comments. Can you review and resolve the comments that are ok now?

Also, I'm not seeing why the checks are failing. Any help on this @PeterC-DLS?

Thanks!!

@phyy-nx
Copy link
Contributor Author

phyy-nx commented Jul 17, 2024

Feedback from Telco: need to deprecate NXdata's scaling_factor and offset instead of replacing them with FIELDNAME_scaling_factor and FIELDNAME_offset

@phyy-nx
Copy link
Contributor Author

phyy-nx commented Jul 17, 2024

Woops I actually had deprecated the original fields. So @paulmillar and @PeterC-DLS this is ready to review. Thanks!

@paulmillar
Copy link

Hi @phyy-nx

Woops I actually had deprecated the original fields. So @paulmillar and @PeterC-DLS this is ready to review. Thanks!

My earlier comment has been addressed.

I've added a NON-BLOCKING comment, meant as a general observation rather than a comment about this particular pull request.

The pull request looks reasonable to me.

@prjemian
Copy link
Contributor

prjemian commented Aug 2, 2024

@paulmillar Once this PR is resolved, visibility of your non-blocking comment will drop. I suggest elevating that visibility by making your comment a new issue (use the GitHub feature).
image

@paulmillar
Copy link

Hi @prjemian

@paulmillar Once this PR is resolved, visibility of your non-blocking comment will drop. I suggest elevating that visibility by making your comment a new issue

Thanks. I've followed your suggestion and created #1400.

- NX_FLOAT -> NX_NUMBER for scaling_factor and offset
- Sync language with number #1396 for the FIELDNAME convention
@phyy-nx
Copy link
Contributor Author

phyy-nx commented Aug 20, 2024

I think I've addressed all the comments @paulmillar @PeterC-DLS. If folks agree, can you resolve your comments and approve it? Approving here doesn't mean merging as it likely needs a vote. Just that it's ready for a vote.

Copy link
Contributor

@PeterC-DLS PeterC-DLS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@phyy-nx
Copy link
Contributor Author

phyy-nx commented Aug 21, 2024

This PR is ready for a NIAC vote. Please vote with an emoji on this comment. Options include thumbs up for yes, thumbs down for no, and anything else to abstain. Thank you. Voting will close in two weeks.

@phyy-nx
Copy link
Contributor Author

phyy-nx commented Sep 5, 2024

Hello all, the vote has passed with 15 in favor. Thanks for the reviews and attention.

@phyy-nx phyy-nx merged commit 9059815 into main Sep 5, 2024
4 checks passed
@phyy-nx phyy-nx deleted the data_scalaroffset branch September 5, 2024 21:13
phyy-nx added a commit to dials/nxmx that referenced this pull request Sep 13, 2024
phyy-nx added a commit to cctbx/dxtbx that referenced this pull request Sep 13, 2024
New parameter as of nexusformat/definitions#1343. This is not full support of the parameter or of data_offset, but it's one of the use cases needed, namely reading a single gain value
ndevenish pushed a commit to dials/nxmx that referenced this pull request Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rework NXdata scaling_factor and offset
4 participants