NOTICE 2023-04-24: DO NOT USE. These document is out of date. See latest version: https://bcda-aps.github.io/bluesky_training/howto/first_steps_guide.html. This repository will be archived by 2023-09-01. It should not be used for new work. All content of this repository (environment files, training documents, reference material) is migrating to https://bcda-aps.github.io/bluesky_training/
The instrument package defines the features of your equipment for use with data
acquisition using the Bluesky framework. It is structured as a Python
package
so that it will be easy to use with the Python import command, like much other
Python software.
There are many ways to define your equipment so consider this guide as a reference for ideas rather than a set of requirements.
Contents
The basic structure is organized into components of the Bluesky framework. All these directories may contain python file(s).
instrument/ <-- all aspects of the equipment
callbacks/ <-- your custom Python code that responds to RunEngine documents or other
devices/ <-- your equipment and its connections with EPICS
framework/ <-- Configure the Bluesky framework
mpl/ <-- Configure MatPlotLib for pltting in a console or Jupyter notebook
plans/ <-- your custom measurement actions (_bluesky plans_)
utils/ <-- (utilities) other Python code that does not fit above
Each directory has a __init__.py file (which can be empty). The very
existence of this file informs Python that the directory is a Python package
which means that it can be imported. The __init__.py file can be used to control
what is imported and the order in which that import is sequenced.
In any of the Python files, the
__all__ symbol is a Python list
that identifies which Python symbols from that file will be imported by default.
(Any Python symbol may be imported by name, this just controls the more general
case.)
To add an EPICS motor to your
instrument package, edit the file instrument/devices/motors.py. You'll need
the EPICS PV for the motor (example is ioc:m1). There may be comments in your
motors.py file to give you a suggestion of the correct command. Add this line
after the various import lines (where the EPICS motor PV is the first
argument):
m1 = EpicsMotor("ioc:m1", name="m1", labels=("motor",))The left side (LHS) assigns the new (ophyd) EpicsMotor() object to the symbol
m1. Follow the rules for naming Python
symbols. By
convention, the name= keyword has the same value as the LHS. The keyword
labels=() is an optional
tuple.
When the motor value is added to the tuple, then this motor will show up in
the report provided by the
%wa
(bluesky IPython) magic command.
CAUTION: Avoid certain names such as
del
and
lambda
since these are reserved Python names with very important meanings.
Be sure to add this new motor symbol to the __all__ list near the top of the file.
Once you add motors to the instrument/devices/motors.py file, you should
enable its automatic import by removing the comment from the line in
instrument/devices/__init__.py:
change
# from .motors import *to be
from .motors import *Often, an instrument will use a single scaler to collect pulse signals from various counting detectors aush as ionization chambers, photodiodes, and scintillation counters. These are coordinated by the EPICS scaler record.
There are two different ophyd objects to describe an EPICS
scaler.
We use the ScalerCH here since it uses the user-defined scaler record channel
names with the acquired data.
Like the motor example above, you'll need the EPICS scaler PV name. Edit the
file instrument/devices/scaler.py and change the ScalerCH line. There are
additional lines if your scaler has predefined names you wish to call in other
Python code. Otherwise, comment out these lines by placing a # character at
the start of each line.
The detectors label is used by the %wa magic for reporting and also the
%ct for convenient counting from the command line. The scalers label helps
to distinguish this type of detector from, for example, area detectors.
Again, edit the __all__ symbol with any new symbol names and enable the import
of scaler.py by editing instrument/devices/__init__.py.
Once many motors have been entered, it might be convenient to group them in
terms of a common structure. Consider a sample X-Y stage might have motors
sample_x and sample_y. These can be grouped together into an ophyd
Device. Also note that a
Device can contain other Devices, in addition to Signals, such as the
ApsMachineParametersDevice.
On ophyd Device organizes one or more ophyd Signals and/or Devices into a larger structure, for grouping or to provide custom controls.
Device Example
Consider this hypothetical case| motor description | EPICS PV |
|---|---|
| sample x motion | ioc:m11 |
| sample y motion | ioc:m12 |
We'll make a new file (don't forget to import the new file in __init__.py)
with these contents (logging and other parts omitted for clarity, follow the
example in motors.py for these.):
__all__ = ["sample_stage", ]
from ophyd import Device, EpicsMotor
from ophyd import Component
class StageXY(Device):
x = Component(EpicsMotor, 'm11', labels=("motor",))
y = Component(EpicsMotor, 'm12', labels=("motor",))
sample_stage = StageXY('ioc:', name='sample_stage')The first argument to our StageXY() is the common prefix for the EPICS PV.
Note also that we do not have to provide the name= keyword for the Components
since ophyd can determine the name as each Component is added into the Device.
Only at the outermost level must the name= keyword be provided. Each of the
components provides the remaining part of the EPICS PV.
| motor description | EPICS PV | ophyd symbol | data name |
|---|---|---|---|
| sample x motion | ioc:m11 |
sample_stage.x |
sample_stage_x |
| sample y motion | ioc:m12 |
sample_stage.y |
sample_stage_y |
Use the ophyd name in your Python code. The data name is how the values are labeled in the data. (There's a reason for this difference beyond this scope.)
NOTE: Once you have used a PV in a custom Device, remove it from any other file such as motors.py. The adminition from the ophyd developers is that you connect a PV once and only once.
EPICS Area Detectors are implemented as custom ophyd Devices that subclass (use) various standard parts from ophyd.
See these examples:
- Dectris Pilatus (with explanations)
- Perkin-Elmer (no explanations, just example code)
The examples so far only begin to demonstrate the variety of Device customizations. Consult the various instrument configurations on the wiki and the apstools package for more device examples.
TIP: Organize your custom Device code into separate files to make it easy to manage. Don't forget to edit the __add__ symbol and also edit the __init__.py file so the new code is imported automatically.
Plans describe your custom sequence of actions. They can be a complete plan that acquires a run or just part of a measurement sequence.
Consult the various instrument configurations on the wiki and the apstools package for more plan examples.
If you are translating PyEpics code to Bluesky plans, consult this guide.
Keep in mind that a plan should not call code that blocks execution of the bluesky
RunEngine from conducting its periodic background actions. One such example is
the sleep()
action
sometimes used to control sequencing of events. More discussion of blocking
is provided in the context of how the RunEngine processes its
Messages.
One custom RunEngine callback has been added to the instrument package
template for the APS. The SPEC file writer
callback
(added before the callbacks/ subpackage was added) is configured in
instrument/framework/callbacks.py. It writes bluesky runs to a text file in
the format of the SPEC data acquisition software.
Different from SPEC (which appends the file line-by-line as data is acquired),
the SPEC callback here appends the file once the run's stop document is
received.
If you do not want SPEC files to be written, then comment out the associated
import in instrument/framework/__init__.py.
A strength of the Bluesky framework is the coordination in a database of acquired data with metadata about the acquisition. This enables a variety of data science after the measurement is complete.
The template instrument package installed initially
configures (in instrument/framework/metadata.py) some simple information that is added to every
bluesky run (in dictionary RE.md):
| key | meaning |
|---|---|
beamline_id |
value of BEAMLINE |
instrument_name |
value of INSTRUMENT |
proposal_id |
(default is comissioning) You should modify this for each proposal |
pid |
process identifier (for diagnostics and logging) |
login_id |
user and workstation collecting the data (for diagnostics and logging) |
versions |
version numbers of various Python packages (for diagnostics and logging) |
The motivation for adding any metadata to a measurement is to use that later as search keys. Sample information, experimenters, proposal and safety form terms: all could be useful later in retrieving related bluesky runs from the database.
Consult this guide for more information about recording metadata. Configurations can be for all runs, specific runs, or even specific types of runs. The system for adding metadata is versatile.
Sometimes, a user may need experiment-specific code that is not for general
inclusion in the instrument package. Components from the instrument package
may be imported from such custom user code by writing a file in the working
directory with the support. The file can be loaded and run in an IPython
session with the %run -i filename.py
magic command.
Example User Code
Here is a contrived example of custom user code.
First, go to a new working directory before starting a bluesky session:
cd /tmp
mkdir demo
cd /tmp/demo
Save this "custom user code" to local file /tmp/demo/my_code.py:
from instrument.devices import m1
print(f"m1 is now at {m1.position:.3f}")Start the bluesky session:
(base) prjemian@zap:/tmp/demo$ blueskyZap.sh
Python 3.8.5 (default, Sep 4 2020, 07:30:14)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.
IPython profile: bluesky
Activating auto-logging. Current session state plus future input saved.
Filename : /tmp/demo/.logs/ipython_console.log
Mode : rotate
Output logging : True
Raw input log : False
Timestamping : True
State : active
I Sun-15:20:22 - ############################################################ startup
I Sun-15:20:22 - logging started
I Sun-15:20:22 - logging level = 10
I Sun-15:20:22 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/collection.py
I Sun-15:20:22 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/mpl/console.py
I Sun-15:20:22 - bluesky framework
I Sun-15:20:22 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/framework/check_python.py
I Sun-15:20:22 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/framework/check_bluesky.py
I Sun-15:20:23 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/framework/initialize.py
I Sun-15:20:23 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/framework/metadata.py
I Sun-15:20:23 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/framework/callbacks.py
I Sun-15:20:23 - writing to SPEC file: /tmp/demo/20210221-152023.dat
I Sun-15:20:23 - >>>> Using default SPEC file name <<<<
I Sun-15:20:23 - file will be created when bluesky ends its next scan
I Sun-15:20:23 - to change SPEC file, use command: newSpecFile('title')
I Sun-15:20:23 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/devices/motors.py
I Sun-15:20:23 - /home/prjemian/.ipython/profile_bluesky/startup/instrument/devices/scaler.py
In [1]:
Run the my_code.py file:
In [1]: %run -i my_code.py
m1 is now at 3.000
In [2]: