Skip to content

Commit

Permalink
Merge branch 'main' into pipeline
Browse files Browse the repository at this point in the history
  • Loading branch information
sneakers-the-rat committed Dec 13, 2024
2 parents 761806e + 920d795 commit 9239c66
Show file tree
Hide file tree
Showing 139 changed files with 2,889 additions and 937 deletions.
4 changes: 2 additions & 2 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
MINISCOPE_IO_BASE_DIR="~/.config/miniscope_io"
MINISCOPE_IO_LOGS__LEVEL="INFO"
MIO_BASE_DIR="~/.config/mio"
MIO_LOGS__LEVEL="INFO"
33 changes: 33 additions & 0 deletions .github/workflows/docs-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Build and test documentation

on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:

jobs:
test_docs:
runs-on: ubuntu-latest

steps:
- name: Check out repository
uses: actions/checkout@v3

- name: Set up python
uses: actions/setup-python@v4
with:
python-version: "3.12"
cache: "pip"

- name: Install dependencies
run: pip install -e .[docs] pytest-md

- name: Build docs
working-directory: docs
env:
SPHINXOPTS: "-W --keep-going"
run: make html
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# miniscope-io
# mio: Miniscope IO

[![PyPI - Version](https://img.shields.io/pypi/v/miniscope-io)](https://pypi.org/project/miniscope_io/)
[![PyPI - Version](https://img.shields.io/pypi/v/mio)](https://pypi.org/project/mio/)
[![Documentation Status](https://readthedocs.org/projects/miniscope-io/badge/?version=latest)](https://miniscope-io.readthedocs.io/en/latest/?badge=latest)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/miniscope-io)
![PyPI - Status](https://img.shields.io/pypi/status/miniscope-io)
[![Coverage Status](https://coveralls.io/repos/github/Aharoni-Lab/miniscope-io/badge.svg?branch=main)](https://coveralls.io/github/Aharoni-Lab/miniscope-io?branch=main)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/mio)
![PyPI - Status](https://img.shields.io/pypi/status/mio)
[![Coverage Status](https://coveralls.io/repos/github/Aharoni-Lab/mio/badge.svg?branch=main)](https://coveralls.io/github/Aharoni-Lab/mio?branch=main)

Generic i/o interfaces for miniscopes.

Expand All @@ -21,4 +21,4 @@ restrictive licenses. Those licenses can be found
in the `LICENSE` files in the respective directories
containing the unmodified source material

* `miniscope_io/vendor/opalkelly`
* `mio/vendor/opalkelly`
2 changes: 1 addition & 1 deletion docs/api/bit_operation.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# bit operation

```{eval-rst}
.. automodule:: miniscope_io.bit_operation
.. automodule:: mio.bit_operation
:members:
:private-members:
```
2 changes: 1 addition & 1 deletion docs/api/exceptions.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# exceptions

```{eval-rst}
.. automodule:: miniscope_io.exceptions
.. automodule:: mio.exceptions
:members:
:undoc-members:
```
12 changes: 0 additions & 12 deletions docs/api/formats/index.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/api/formats/sdcard.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/api/formats/stream.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/api/io.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# io

```{eval-rst}
.. automodule:: miniscope_io.io
.. automodule:: mio.io
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/logging.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# logging

```{eval-rst}
.. automodule:: miniscope_io.logging
.. automodule:: mio.logging
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/buffer.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# buffer

```{eval-rst}
.. automodule:: miniscope_io.models.buffer
.. automodule:: mio.models.buffer
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/config.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# config

```{eval-rst}
.. automodule:: miniscope_io.models.config
.. automodule:: mio.models.config
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/data.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# data

```{eval-rst}
.. automodule:: miniscope_io.data
.. automodule:: mio.data
:members:
:undoc-members:
```
9 changes: 8 additions & 1 deletion docs/api/models/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,20 @@
# models

Pydantic models used throughout `miniscope_io`.
Pydantic models used throughout `mio`.

These models should be kept as generic as possible, and any refinements
needed for a specific acquisition class should be defined within that
module, inheriting from the relevant parent class. Rule of thumb:
keep what is common common, and what is unique unique.



```{eval-rst}
.. automodule:: mio.models
:members:
:undoc-members:
```

```{toctree}
:maxdepth: 1
Expand Down
2 changes: 1 addition & 1 deletion docs/api/models/mixins.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# mixins

```{eval-rst}
.. automodule:: miniscope_io.models.mixins
.. automodule:: mio.models.mixins
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/models.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# models

```{eval-rst}
.. automodule:: miniscope_io.models.models
.. automodule:: mio.models.models
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/sdcard.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# sdcard

```{eval-rst}
.. automodule:: miniscope_io.models.sdcard
.. automodule:: mio.models.sdcard
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/models/stream.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# stream

```{eval-rst}
.. automodule:: miniscope_io.models.stream
.. automodule:: mio.models.stream
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/plots/headers.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# headers

```{eval-rst}
.. automodule:: miniscope_io.plots.headers
.. automodule:: mio.plots.headers
:members:
:undoc-members:
```
12 changes: 6 additions & 6 deletions docs/api/stream_daq.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
This module is a data acquisition module that captures video streams from Miniscopes based on the `Miniscope-SAMD-Framework` firmware. The firmware repository will be published in future updates but is currently under development and private.

## Command
After [installation](../guide/installation.md) and customizing [device configurations](stream-dev-config) and [runtime configuration](models/config.md) if necessary, run the command described in [CLI Usage](../guide/cli).
After [installation](../guide/installation.md) and customizing [device configurations](stream-dev-config) and [runtime configuration](models/config.md) if necessary, run the command described in [CLI Usage](../cli/index).

One example of this command is the following:
```bash
$ mio stream capture -c .path/to/device/config.yml -o output_filename.avi -m
```
A window displaying the image transferred from the Miniscope and a graph plotting metadata (`-m` option) should pop up. Additionally, the indexes of captured frames and their statuses will be logged in the terminal. The `MINISCOPE_IO_STREAM_HEADER_PLOT_KEY` defines plotted header fields (see `.env.sample`).
A window displaying the image transferred from the Miniscope and a graph plotting metadata (`-m` option) should pop up. Additionally, the indexes of captured frames and their statuses will be logged in the terminal. The `MIO_STREAM_HEADER_PLOT_KEY` defines plotted header fields (see `.env.sample`).

## Prerequisites
- **Data capture hardware:** Opal Kelly XEM7310-A75 FPGA board (connected via USB)
Expand All @@ -17,10 +17,10 @@ A window displaying the image transferred from the Miniscope and a graph plottin

(stream-dev-config)=
## Device configuration
A YAML file is used to configure Stream DAQ based on the device configuration. The device configuration needs to match the imaging and data capture hardware for proper operation. This file is used to set up hardware, define data formats, and set data preambles. The contents of this YAML file will be parsed into a model [miniscope_io.models.stream](../api/models/stream.md), which then configures the Stream DAQ.
A YAML file is used to configure Stream DAQ based on the device configuration. The device configuration needs to match the imaging and data capture hardware for proper operation. This file is used to set up hardware, define data formats, and set data preambles. The contents of this YAML file will be parsed into a model [mio.models.stream](../api/models/stream.md), which then configures the Stream DAQ.

### FPGA (Opal Kelly) configuration
The `bitstream` field in the device configuration yaml file specifies the image that will be uploaded to the opal kelly board. This file needs to be placed in `miniscope_io.devices`.
The `bitstream` field in the device configuration yaml file specifies the image that will be uploaded to the opal kelly board. This file needs to be placed in `mio.devices`.


#### Bitstream file nomenclature
Expand All @@ -33,7 +33,7 @@ Name format of the bitstream files and directory:
- **ENCODING_POLARITY**: Manchester encoding convention, which corresponds to bit polarity. The current package supports IEEE convention Manchester encoding.

### Example device configuration
Below is an example configuration YAML file. More examples can be found in `miniscope_io.data.config`.
Below is an example configuration YAML file. More examples can be found in `mio.data.config`.

```yaml
# capture device. "OK" (Opal Kelly) or "UART"
Expand Down Expand Up @@ -91,7 +91,7 @@ runtime:
```
```{eval-rst}
.. automodule:: miniscope_io.stream_daq
.. automodule:: mio.stream_daq
:members:
:undoc-members:
```
2 changes: 1 addition & 1 deletion docs/api/utils.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# utils

```{eval-rst}
.. automodule:: miniscope_io.utils
.. automodule:: mio.utils
:members:
:undoc-members:
```
9 changes: 9 additions & 0 deletions docs/cli/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# `config`

See also: [config guide](../guide/config.md)

```{eval-rst}
.. click:: mio.cli.config:config
:prog: mio config
:nested: full
```
6 changes: 6 additions & 0 deletions docs/cli/device.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# `device`

```{eval-rst}
.. click:: mio.cli.main:device
:prog: mio device
```
19 changes: 19 additions & 0 deletions docs/cli/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# CLI Usage

```{toctree}
:maxdepth: 1
config
device
stream
update
```

Refer to the following page for details regarding ``stream_daq`` device config files.

- [stream_daq](../api/stream_daq.md)

```{eval-rst}
.. click:: mio.cli.main:cli
:prog: mio
```
7 changes: 7 additions & 0 deletions docs/cli/stream.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# `stream`

```{eval-rst}
.. click:: mio.cli.stream:stream
:prog: mio stream
:nested: full
```
7 changes: 7 additions & 0 deletions docs/cli/update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# `update`

```{eval-rst}
.. click:: mio.cli.update:update
:prog: mio update
:nested: full
```
7 changes: 4 additions & 3 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
project = "miniscope-io"
copyright = "2023, Jonny"
author = "Jonny, Takuya"
release = _version("miniscope-io")
html_title = "miniscope-io"
release = _version("mio")
html_title = "mio"

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
Expand All @@ -32,8 +32,10 @@
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
"sphinx_click",
"sphinx_design",
"sphinxcontrib.autodoc_pydantic",
"sphinxcontrib.mermaid",
"sphinxcontrib.programoutput",
]

templates_path = ["_templates"]
Expand All @@ -44,7 +46,6 @@
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = "furo"
html_static_path = ["_static"]

intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
Expand Down
4 changes: 2 additions & 2 deletions docs/guide/capture/wireless.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@

## Header Values and Expected Transitions
See following docs for the basic structure.
- `miniscope_io.models.buffer.BufferHeaderFormat`
- `miniscope_io.models.stream.StreamBufferHeaderFormat`
- `mio.models.buffer.BufferHeaderFormat`
- `mio.models.stream.StreamBufferHeaderFormat`

Device specific notes are listed below.
- **`preamble`**: 32-bit preamble for detecting the beginning of each buffer. The [`preamble`](../../api/stream_daq.md) in the device config needs to match the preamble defined in firmware.
Expand Down
Loading

0 comments on commit 9239c66

Please sign in to comment.