Introduction to the Lossless Pipeline

This EEG processing pipeline is especially useful for the following scenarios:

• You want to keep your EEG data in a continuous state, allowing you the flexibility to epoch your data at a later stage.
• You are part of a research team or community that shares a common dataset, and you want to process the data once in a way that can be used for multiple analyses (i.e., one analysis can segment the cleaned data into 10-second epochs and filter the data betweeen 1-30Hz, while another analysis can use 1-second epochs with no filter, etc.)
• You want to be able to do a hands on review of the pre-processing results for each file.

📘 Installation and usage instructions

The development version can be installed from GitHub with

$ git clone git@github.com:lina-usc/pylossless.git
$ pip install --editable ./pylossless

for an editable installation, or simply with

$ pip install git+https://github.com/lina-usc/pylossless.git

for a static version.

Please find the full documentation at pylossless.readthedocs.io.

▶️ Running the pyLossless Pipeline

Below is a minimal example that runs the pipeline one of MNE's sample files.

import pylossless as ll 
import mne
fname = mne.datasets.sample.data_path() / 'MEG' / 'sample' /  'sample_audvis_raw.fif'
raw = mne.io.read_raw_fif(fname, preload=True)

config = ll.config.Config()
config.load_default()

pipeline = ll.LosslessPipeline(config=config)
pipeline.run_with_raw(raw)

Once it is completed, You can see what channels and times were flagged:

print(pipeline.flagged_chs)
print(pipeline.flagged_epochs)

Once you are ready, you can save your file in its lossless state:

pipeline.save(pipeline.get_derivative_path(bids_path), overwrite=True)

To get a cleaned version, you can use a RejectionPolicy object to apply these annotations to your raw object. This is a lossy operation:

rejection_policy = ll.RejectionPolicy()
cleaned_raw = rejection_policy.apply(pipeline)

👩‍💻 Dashboard Review

After running the Lossless pipeline, you can launch the Quality Control Review (QC) dashboard to review the pipeline's decisions on each file! You can flag additional channels, times and components, and edit flags made by the pipeline.

First install the dashboard requirements

$ cd ./path/to/pylossless/on/your/computer
$ pip install --editable .[dash]

$ pylossless_qc

▶️ Example HPC Environment Setup

If you are a Canadian researcher working on an HPC system such as Narval:

module load python/3.10

# Build the virtualenv in your homedir
cd ~
virtualenv --no-download eeg-env
source eeg-env/bin/activate

pip install --no-index mne
pip install --no-index pandas
pip install --no-index xarray
pip install --no-index pyyaml
pip install --no-index sklearn
pip install mne_bids
pip install edfio
pip install openneuro-py

pip install mne-icalabel

pip install --no-deps pylossless

# Verify that the package has installed correct with an import
python -c 'import pylossless'