Skip to content

Commit

Permalink
Autosave support (#163)
Browse files Browse the repository at this point in the history
Add support for saving and restoring pv fields with autosave
  • Loading branch information
jsouter authored Sep 13, 2024
1 parent dcccdd6 commit 3b0bef0
Show file tree
Hide file tree
Showing 16 changed files with 1,181 additions and 21 deletions.
38 changes: 24 additions & 14 deletions .github/workflows/code.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ jobs:
runs-on: "ubuntu-latest"
steps:
- name: Checkout Source
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Install Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.7"

Expand All @@ -34,7 +34,7 @@ jobs:
runs-on: "ubuntu-latest"
steps:
- name: Checkout Source
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
# require history to get back to last tag for version number of branches
fetch-depth: 0
Expand All @@ -44,7 +44,7 @@ jobs:
run: pipx run build --sdist .

- name: Upload Sdist
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: dist
path: dist/*
Expand All @@ -56,6 +56,11 @@ jobs:
os: [ubuntu-latest, windows-latest, macos-latest]
python: [cp37, cp38, cp39, cp310]

exclude:
# MacOS 14.4.1 for arm64 doesn't support Python < 3.8
- os: macos-latest
python: "cp37"

include:
# Put coverage and results files in the project directory for mac
- os: macos-latest
Expand All @@ -69,22 +74,27 @@ jobs:
- os: ubuntu-latest
cov_file: /output/coverage.xml
results_file: /output/pytest-results.xml
# MacOS 13 required for Python < 3.8
- os: macos-13
python: "cp37"
cov_file: "{project}/dist/coverage.xml"
results_file: "{project}/dist/pytest-results.xml"

name: build/${{ matrix.os }}/${{ matrix.python }}
runs-on: ${{ matrix.os }}

steps:
- name: Checkout Source
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
# require history to get back to last tag for version number of branches
fetch-depth: 0
submodules: true

- name: Install Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.7"
python-version: "3.12"

- name: Install Python Dependencies
# Pin cibuildwheel due to https://github.com/pypa/cibuildwheel/issues/962
Expand All @@ -106,20 +116,20 @@ jobs:
CIBW_SKIP: "*-musllinux*" # epicscorelibs doesn't build on musllinux platforms

- name: Upload Wheel
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: dist
name: dist-${{ matrix.os }}-${{ matrix.python }}
path: dist/softioc*

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v2
uses: codecov/codecov-action@v4
with:
name: ${{ matrix.os }}/${{ matrix.python }}
directory: dist

- name: Upload Unit Test Results
if: always()
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: Unit Test Results (${{ matrix.os }}-${{ matrix.python }})
path: dist/pytest-results.xml
Expand All @@ -132,7 +142,7 @@ jobs:

steps:
- name: Download Artifacts
uses: actions/download-artifact@v2
uses: actions/download-artifact@v4
with:
path: artifacts

Expand All @@ -152,7 +162,7 @@ jobs:
runs-on: ${{ matrix.os }}

steps:
- uses: actions/download-artifact@v2
- uses: actions/download-artifact@v4
with:
name: dist
path: dist
Expand All @@ -167,7 +177,7 @@ jobs:
# upload to PyPI and make a release on every tag
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
steps:
- uses: actions/download-artifact@v2
- uses: actions/download-artifact@v4
with:
name: dist
path: dist
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Unreleased_
-----------

Added:

- `Add autosave support to all records and record fields <../../pull/163>`_
- `Add int64In/Out record support <../../pull/161>`_
- `Enable setting alarm status of Out records <../../pull/157>`_
- `Adding the non_interactive_ioc function <../../pull/156>`_
Expand Down
90 changes: 90 additions & 0 deletions Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 34 additions & 0 deletions docs/examples/example_autosave_ioc.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
from softioc import autosave, builder, softioc
import cothread

# Set the record prefix
builder.SetDeviceName("MY-DEVICE-PREFIX")

# Create records, set some of them to autosave, also save some of their fields

builder.aOut("AO", autosave=True)
builder.aIn("AI", autosave=["PREC", "EGU"])
builder.boolIn("BO")
builder.WaveformIn("WAVEFORMIN", [0, 0, 0, 0], autosave=True)
with autosave.Autosave(["VAL", "LOPR", "HOPR"]):
builder.aOut("AUTOMATIC-AO", autosave=["EGU"])
seconds = builder.longOut("SECONDSRUN", autosave=True)

autosave.configure(
directory="/tmp/autosave-data",
name="MY-DEVICE-PREFIX",
save_period=5.0
)

builder.LoadDatabase()
softioc.iocInit()

# Start processes required to be run after iocInit
def update():
while True:
cothread.Sleep(1)
seconds.set(seconds.get() + 1)

cothread.Spawn(update)

softioc.interactive_ioc(globals())
76 changes: 76 additions & 0 deletions docs/how-to/use-autosave-in-an-ioc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
Use `softioc.autosave` in an IOC
================================

`../tutorials/creating-an-ioc` shows how to create a pythonSoftIOC.


Example IOC
-----------

.. literalinclude:: ../examples/example_autosave_ioc.py

Records are instantiated as normal and configured for automatic loading and
periodic saving to a backup file with use of the keyword argument ``autosave``.
``autosave`` resolves to a list of strings, which are the names of fields to be
tracked by autosave. By default ``autosave=False``, which disables autosave for that PV.
Setting ``autosave=True`` is equivalent to passing ``["VAL"]``. Note that ``"VAL"`` must be
explicitly passed when tracking other fields, e.g. ``["VAL", "LOPR", "HOPR"]``.
``autosave`` can also accept a single string field name as an argument.

The field values get written into a yaml-formatted file containing key-value pairs.
By default the keys are the same as the full PV name, including any device name specified
in :func:`~softioc.builder.SetDeviceName()`.

Autosave is disabled until :func:`~softioc.autosave.configure()` is called. The first two arguments,
``directory`` and ``name`` are required. Backup files are periodically written into
``directory`` with the name ``<name>.softsav`` every ``save_period`` seconds,
set to 30.0 by default. The directory must exist, and should be configured with the appropriate
read/write permissions for the user running the IOC.

IOC developers should only need to interface with autosave via the :func:`~softioc.autosave.configure()`
method and the ``autosave`` keyword argument. Alternatively,
PVs can be instantiated inside the :class:`~softioc.autosave.Autosave()` context manager, which
automatically passes the ``autosave`` argument to any PVs created
inside the context manager. If any fields are already specified by the ``autosave`` keyword
argument of a PV's initialisation call the lists of fields to track get combined.
All other module members are intended for internal use only.

In normal operation, loading from a backup is performed once during the
:func:`~softioc.builder.LoadDatabase()` call and periodic saving to the backup file begins when
:func:`~softioc.softioc.iocInit()` is called, provided that any PVs are configured to be saved.
Currently, manual loading from a backup at runtime after ioc initialisation is not supported.
Saving only occurs when any of the saved field values have changed since the last save.
Users are discouraged from manually editing the backup files while the
IOC is running so that the internal state of the autosave thread is consistent with
the backup file.

If autosave is enabled and active, a timestamped copy of the latest existing autosave backup file is created
when the IOC is restarted, e.g. ``<name>.softsav_240717-095004`` (timestamps are in the format yymmdd-HHMMSS).
If you only wish to store one backup of the autosave file at a time, ``timestamped_backups=False``
can be passed to :func:`~softioc.autosave.configure()` when it is called, this will create a backup file
named ``<name>.softsav.bu``. To disable any autosaving, comment out the
:func:`~softioc.autosave.configure()` call or pass it the keyword argument
``enabled=False``.

The resulting backup file after running the example IOC for about 30 seconds is the following:

.. code-block::
MY-DEVICE-PREFIX:AI.EGU: ''
MY-DEVICE-PREFIX:AI.PREC: '0'
MY-DEVICE-PREFIX:AO: 0.0
MY-DEVICE-PREFIX:AUTOMATIC-AO: 0.0
MY-DEVICE-PREFIX:AUTOMATIC-AO.EGU: ''
MY-DEVICE-PREFIX:AUTOMATIC-AO.HOPR: '0'
MY-DEVICE-PREFIX:AUTOMATIC-AO.LOPR: '0'
MY-DEVICE-PREFIX:SECONDSRUN: 29
MY-DEVICE-PREFIX:WAVEFORMIN: [0, 0, 0, 0]
If the IOC is stopped and restarted, the SECONDSRUN record will load its saved
value of 29 from the backup.
All non-VAL fields are stored as strings. Waveform type records holding arrays
are cast into lists before saving.

This example IOC uses cothread, but autosave works identically when using
an asyncio dispatcher.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ Table Of Contents
:maxdepth: 1

how-to/use-asyncio-in-an-ioc
how-to/use-autosave-in-an-ioc
how-to/make-publishable-ioc
how-to/read-data-from-ioc
how-to/use-soft-records
Expand Down
Loading

0 comments on commit 3b0bef0

Please sign in to comment.