Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demark plan vs stub, move orphaned plans into dodal #793

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

DiamondJoseph
Copy link
Contributor

@DiamondJoseph DiamondJoseph commented Sep 19, 2024

Bluesky distinguishes between plans: complete experimental proceedures, which open and close data collection runs, which may be part of a larger plan that collect data multiple times, but that might also be run alone to collect data, and plan_stubs: which do not create & complete data collection runs and are either isolated behaviours or building blocks for plans.

In order to make it clearer when a MsgGenerator can be safely used without considering the enclosing run (as opening a run whilst in a run without explicitly passing a RunID is likely to cause both runs to fail), when it is required to manage a run and when running a procedure will create data documents, we should adopt this standard.

This change also adds the count and spec_scan plan previously from dls_bluesky_core, as remove this as a requirement for blueapi is long overdue and this is the last remnants of that module. count is a basic collection of a set of devices, equivalent to Mapping GDA's StaticScan. spec_scan is analogous to GDA's mscan command, constructing a generic N-dimensional trajectory for scanning.

After these plans have been moved, the default blueapi context can be configured to import as a plans module dodal.plans and dodal.plan_stubs.wrapped: this will only expose those stubs that should be runnable by a user from the client.

Checks for reviewer

  • Would the PR title make sense to a scientist on a set of release notes

@DiamondJoseph DiamondJoseph force-pushed the move-example-plans branch 2 times, most recently from 7ab3c64 to 019b6ed Compare September 20, 2024 11:56
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.70%. Comparing base (f896461) to head (cb60a6d).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #793      +/-   ##
==========================================
+ Coverage   95.63%   95.70%   +0.06%     
==========================================
  Files         128      132       +4     
  Lines        5271     5326      +55     
==========================================
+ Hits         5041     5097      +56     
+ Misses        230      229       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@DiamondJoseph DiamondJoseph force-pushed the move-example-plans branch 3 times, most recently from fddc20d to 037fe3e Compare October 22, 2024 13:09
@DiamondJoseph DiamondJoseph marked this pull request as ready for review October 23, 2024 12:12
src/dodal/plans/scanspec.py Outdated Show resolved Hide resolved
src/dodal/plan_stubs/__init__.py Outdated Show resolved Hide resolved
tests/plans/test_scanspec.py Outdated Show resolved Hide resolved
@DiamondJoseph DiamondJoseph changed the title Demark behavioural differences between plans and stubs, move orphaned plans into dodal. Demark plan vs stub, move orphaned plans into dodal Oct 24, 2024
Copy link
Contributor

@DominicOram DominicOram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with all the changes that affect MX, thanks for doing those. I have added some comments to the other code but given it's just a copy of existing code and we do not make use of it in MX please consider them optional.

src/dodal/plan_stubs/wrapped.py Show resolved Hide resolved
src/dodal/plan_stubs/wrapped.py Outdated Show resolved Hide resolved

@attach_data_session_metadata_decorator()
@validate_call(config={"arbitrary_types_allowed": True})
def spec_scan(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: I added DiamondLightSource/mx-bluesky#596 as I think we have overlap here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This plan has now moved repositories so many times that the issue has now been lost to time, but it does need improvement. The axes_to_move parameter was a temporary fix and we should try to get rid of it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@DiamondJoseph DiamondJoseph Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we no longer need axes_to_move, I just need to find time to start up blueapi with these plans and ensure we can run a spec just passing the name of the device and no axes_to_move

Copy link
Contributor Author

@DiamondJoseph DiamondJoseph Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, all plans that use a ScanSpec should be constructing their specific spec and passing it to this (for software scans) or an ophyd-async plan to-be-produced for hardware.

Comment on lines +33 to +36
"""Reads from a number of devices.
Wraps bluesky.plans.count(det, num, delay, md=metadata) exposing only serializable
parameters and metadata."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I assume we've discussed with bluesky to try and get this into the core codebase and couldn't? May be worth a link to that discussion in case anyone asks the question again

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a series of mostly offline discussions and should probably be written up as an ADR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does someone involved in the discussion have time to write an issue with some back of the envelope bullet point reasons and saying to put them in an ADR, just before it's lost to time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we need DiamondLightSource/blueapi#474 for the @attach_metadata to be handled in blueapi consistently, then that decorator gets moved back into blueapi and then blueapi can just use bluesky.plans as a plans module (or... the ones we actually want).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case can we have a comment here saying we can remove this when that has been done?

tests/plan_stubs/test_wrapped_stubs.py Outdated Show resolved Hide resolved
tests/plans/test_compliance.py Show resolved Hide resolved
tests/plans/test_scanspec.py Outdated Show resolved Hide resolved
tests/plans/test_wrapped.py Outdated Show resolved Hide resolved
tests/plans/test_wrapped.py Outdated Show resolved Hide resolved
Copy link
Contributor

@DominicOram DominicOram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment but otherwise looks really good, much cleaner tests, thank you!

Comment on lines +33 to +36
"""Reads from a number of devices.
Wraps bluesky.plans.count(det, num, delay, md=metadata) exposing only serializable
parameters and metadata."""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which case can we have a comment here saying we can remove this when that has been done?

Comment on lines +11 to +29
"""
Bluesky distinguishes between `plans`: complete experimental proceedures, which open and
close data collection runs, and which may be part of a larger plan that collect data
multiple times, but may also be run alone to collect data, and `plan_stubs`: which
do not create & complete data collection runs and are either isolated behaviours or
building blocks for plans.

In order to make it clearer when a MsgGenerator can be safely used without considering
the enclosing run (as opening a run whilst in a run without explicitly passing a RunID
is likely to cause both runs to fail), when it is required to manage a run and when
running a procedure will create data documents, we have adopted this standard.

We further impose other requirements on both plans and stubs exported from these modules
to enable them to be exposed in UIs in a consistent way:
- They must have a docstring
- They must not use variadic arguments (*args, **kwargs)
- Plans must and Stubs may have an optional argument for metadata, named "metadata"
- The metadata argument, where present, must be optional with a default of None
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants