Update I24 serial docs

docs/developer/serial-crystallography-on-i24/how-to/environment-setup.rst

@@ -25,7 +25,7 @@ shell commands pointing to the correct scripts/edm locations.
 
 .. code:: bash
 
-   ./path/to/mx-bluesky/deploy/deploy_edm_for_ssx.sh
+   ./path/to/mx-bluesky/utility_scripts/deploy/deploy_edm_for_ssx.sh
 
 Setting the current visit directory
 ===================================
@@ -37,7 +37,7 @@ point to the current visit and then running the command:
 
 .. code:: bash
 
-   ./path/to/mx-bluesky/src/mx_bluesky/i24/serial/set_visit_directory.sh
+   ./path/to/mx-bluesky/src/mx_bluesky/beamlines/i24/serial/set_visit_directory.sh
 
 Note that the default experiment type for the script setting the
 directory will be ``fixed-target``. In case of an extruder collection,
@@ -46,4 +46,4 @@ the command line.
 
 .. code:: bash
 
-   ./path/to/mx-bluesky/src/mx_bluesky/i24/serial/set_visit_directory.sh extruder
+   ./path/to/mx-bluesky/src/mx_bluesky/beamlines/i24/serial/set_visit_directory.sh extruder

docs/developer/serial-crystallography-on-i24/how-to/run-a-collection.rst

@@ -12,7 +12,7 @@ A couple of entry points have been set up so that:
 
 Before opening the experiment specific edm, each of these entry points
 will start a ``BlueAPI`` server. The configuration used by ``BlueAPI``
-is saved in ``src/mx_bluesky/i24/serial/blueapi_config.yaml``.
+is saved in ``src/mx_bluesky/beamlines/i24/serial/blueapi_config.yaml``.
 
 Detector choice
 ===============
@@ -54,7 +54,9 @@ into a single sub-directory but split into multiple smaller collections.
 
 For a pump-probe experiment, select ``True`` on the dropdown menu in
 ``Data Collection Setup`` and then set the laser dwell and delay times
-in the ``Pump Probe`` panel. **WARNING** This setting requires an
+in the ``Pump Probe`` panel. 
+
+**WARNING** This setting requires a
 hardware change, as there are only 4 outputs on the zebra and they are
 all in use. When using the Eiger the Pilatus trigger cable should be
 used to trigger the light source. When using the pilatus the eiger
@@ -75,7 +77,7 @@ set up the coordinate system.
 
 Before this step remember to reset the scale and skew factors as well as
 the motor directions as needed. Current values are saved in
-``src/mx_bluesky/i24/serial/parameters/fixed_target`` in the
+``src/mx_bluesky/beamlines/i24/serial/parameters/fixed_target`` in the
 ``cs_maker.json`` and ``motor_direction.txt`` files.
 
 1. From the main edm screen open the ``viewer`` and ``moveonclick``.
@@ -146,17 +148,15 @@ set the laser dwell and delay times accordingly.
 For more details on the pump probe settings see `Dynamics and fixed
 targets <https://confluence.diamond.ac.uk/display/MXTech/Dynamics+and+fixed+targets>`__
 
-III - **Save the parameters**
-
-**This step cannot be skipped!**
-
-Once all of the previous steps have been completed - and before running
-a collection - all parameters have to be saved using the
-``Set parameters`` button so that they can be applied to the collection.
-A copy parameter file will be saved along with the chip map (if
-applicable) in the data directory at collection time.
 
-IV - **Run a collection**
+III - **Run a collection**
 
 Once all parameters have been set, press ``Start`` to run the
 collection. A stream log will show what is going on in the terminal.
+
+
+**NOTE** As of version ``1.0.0``, the ``Set parameters`` button has been removed and
+the parameters will now be read from the edm and applied to the collection directly
+once the ``Start`` button is pressed. For previous versions however, the button must
+still be pressed before starting the collection. A copy of the parameter file and chip
+map (if applicable) will still be saved in the data directory at collection time.

docs/developer/serial-crystallography-on-i24/how-to/stage-pmac-moves.rst

@@ -47,7 +47,7 @@ in a couple of different ways.
 
    .. code:: python
 
-      yield from bps.mv(pmac.x, 0, pmac.y, 1)https://github.com/DiamondLightSource/mx-bluesky/wiki/Serial-Crystallography-on-I24#cs_reset-custom-chips
+      yield from bps.mv(pmac.x, 0, pmac.y, 1)
 
 Notes on the coordinate system for a fixed-target collection
 ============================================================

docs/developer/serial-crystallography-on-i24/project-planning/roadmap.rst

@@ -4,29 +4,126 @@ Roadmap
 For a closer look at the ongoing work: `I24ssx
 board <https://github.com/orgs/DiamondLightSource/projects/10/views/2>`__
 
-Ongoing list of things needed:
-
-1. Solution for enabling users to run mx-bluesky instead of old scripts:
-   blocked by permission issues. Preferred solution would be run on
-   beamline kubernetes cluster - in the meantime also loooking into
-   procserv as a possibility. (Fixing this should also allow us to stop
-   using the Pilatus to make directories during an Eiger collection.)
-   Temporary workaround in place for beamline staff, who should be
-   starting to run this for testing/in house beamtimes.
-2. Convert detector set up to use bluesky plans and ophyd_async devices.
-   Investigate using the existing Pilatus in ophyd_async which writes
-   HDF5 instead of CBFs (may want to make a CBF-writing Pilatus).
-3. Start looking into moving away from edm screens towards a web-based
-   GUI.
-4. Improve alignment of chip: get it to work correctly for multiple
+Ongoing list of TODOs:
+
+1. Improve alignment of chip: get it to work correctly for multiple
    zooms.
 
+   - Close to being completed (see issue #44)
+
+2. Refactor and fix the logger.
+3. Use callbacks for file IO, eg. to write parameter and map files, nexus writer and ispyb deposition.
+4. Solution for enabling users to run mx-bluesky instead of old scripts: blocked by permission issues.
+
+   - This should also allow us to stop using the Pilatus to make directories during an Eiger collection.
+   - The preferred permanent solution is to run mx-bluesky on the beamline kubernetes cluster. I24 should be due to get on in shutdown 4 2024.
+   - Other possibility is to run the ``blueapi`` server on procServ. An example of this has now been set up on ws002 on the beamline and is being tested.
+   - A temporary workaround in place for beamline staff, who should be starting to run this for testing/in house beamtimes. Staff have started using this set up in September 24.
+
+5. Convert detector set up to use bluesky plans and ophyd_async devices.
+
+   - Eiger device in dodal needs to be converted to ophyd_async and updated to work with different Eigers on several beamlines. This work is dependent on other work out of the scope of this project, see `Dodal#700 <https://github.com/DiamondLightSource/dodal/issues/700>`__ and linked issues.
+   - Investigate using the existing Pilatus in ophyd_async which writes HDF5 instead of CBFs, but we may want to make a CBF-writing Pilatus. However, the Pilatus detector is due to be removed soon.
+
+6. Start looking into moving away from edm screens towards a web-based GUI.
+
+   - Prepare generic react components to match the features in the edms.
+   - Components for things currently managed by general purpose PVs (eg. map).
+   - OAV viewer.
+
+7. Start integrating Panda, at least for fixed-target collections. 
+8. Implementation of serial tools to be used at XFELS.
+
+   - Reinstate removed code from sacla and move it to bluesky.
+   - Add any plans/devices that might be needed for other locations.
+
+9. Reinstate full mapping code using bluesky.
+
 (TBC…)
 
+
+.. list-table:: Rough Roadmap
+   :widths: 30 30 15
+   :header-rows: 1
+
+   * - Work Ongoing
+     - Rough Timeline
+     - Completed
+   * - Fully test extruder collections
+     - Nov. 24
+     - :material-regular:`pending;2em`
+   * - Fix permissions and allow for user collections
+     - Dec. 24 / Jan. 25
+     - :material-regular:`pending;2em`
+   * - Convert the current detector set up code to bluesky plans using the device
+     - Dependent on `FastCS Eiger issues <https://github.com/bluesky/ophyd-async/issues?q=is%3Aissue+is%3Aopen+eiger>`__ being completed
+     - :material-regular:`pending;2em`
+   * - Set up callback for nexus writing
+     - Dec. 24
+     - :material-regular:`pending;2em`
+   * - Set up callback for ispyb deposition
+     - Dec. 24
+     - :material-regular:`pending;2em`
+   * - Set up callback for parameter and map file I/O
+     - Dec. 24
+     - :material-regular:`pending;2em`
+   * - Refactor logger
+     - Nov. 24
+     - :material-regular:`pending;2em`
+   * - Improve current alignment (moveonclick)
+     - Nov. 24
+     - :material-regular:`pending;2em`
+   * - Set up a PV backend, eg. ``pvws``, for web GUI on the beamline.
+     - Jan. 25
+     - :material-regular:`pending;2em`
+   * - Prepare React components to switch from EDM to a web GUI
+     - Dec. 24 / Jan. 25
+     - :material-regular:`pending;2em`
+   * - Move the OAV viewer to a web GUI
+     - Jan. 25 / Feb. 25
+     - :material-regular:`pending;2em`
+
+
+Experiment types required
+=========================
+
+-  Extruder
+
+   -  Standard
+   -  Pump probe
+
+-  Fixed target (probably about 80-85% of serial on I24)
+
+   -  Standard chip collection – option for multiple exposures in each
+      spot
+   -  Pump probe - see for short description
+      https://confluence.diamond.ac.uk/display/MXTech/Dynamics+and+fixed+targets
+
+      -  Short delays
+      -  Excite and visit again
+      -  Long delays with fast shutter opening/closing
+
+-  (Future) Fixed target with rotation at each “window” (Preliminary
+   work done by beamline staff on the PMAC program
+   https://confluence.diamond.ac.uk/display/MXTech/Grids+with+rotations)
+
+Details of zebra settings for each type:
+https://confluence.diamond.ac.uk/display/MXTech/Zebra+settings+I24
+
+Note that most of the set up for the fixed target is actually done by
+the PMAC via PMAC strings.
+
+
+
+--------------
+
+Old roadmap for reference
+
+
 +---------------------------------------+----------------+---------------------------------+
 |             Work Ongoing              | Rough Timeline |            Completed            |
 +=======================================+================+=================================+
-| Document how to set up the current    | Ongoing        | :material-regular:`pending;2em` |
+| Document how to set up the current    | Ongoing        | :material-regular:`check;2em`   |
 | visit, deploy the edm screens and run |                |                                 |
 | a simple collection                   |                |                                 |
 +---------------------------------------+----------------+---------------------------------+
@@ -71,34 +168,3 @@ Ongoing list of things needed:
 | Tidy up original code and add some    | Summer 23      | :material-regular:`check;2em`   |
 | tests                                 |                |                                 |
 +---------------------------------------+----------------+---------------------------------+
-
---------------
-
-Experiment types required
-=========================
-
--  Extruder
-
-   -  Standard
-   -  Pump probe
-
--  Fixed target (probably about 80-85% of serial on I24)
-
-   -  Standard chip collection – option for multiple exposures in each
-      spot
-   -  Pump probe - see for short description
-      https://confluence.diamond.ac.uk/display/MXTech/Dynamics+and+fixed+targets
-
-      -  Short delays
-      -  Excite and visit again
-      -  Long delays with fs opening/closing
-
--  (Future) Fixed target with rotation at each “window” (Preliminary
-   work done by beamline staff on the PMAC program
-   https://confluence.diamond.ac.uk/display/MXTech/Grids+with+rotations)
-
-Details of zebra settings for each type:
-https://confluence.diamond.ac.uk/display/MXTech/Zebra+settings+I24
-
-Note that most of the set up for the fixed target is actually done by
-the PMAC via PMAC strings.