Add tutorials for electrophysiology and fiber photometry

src/constantinople_lab_to_nwb/schierek_embargo_2024/interfaces/schierek_embargo_2024_processedbehaviorinterface.py

@@ -107,7 +107,7 @@ def add_to_nwbfile(
             if column in data:
                 data[column] = list(np.array(data[column]).astype(bool))
 
-        columns_to_add = data.keys()
+        columns_to_add = list(data.keys())
         if column_name_mapping is not None:
             columns_to_add = [column for column in column_name_mapping.keys() if column in data.keys()]
 

src/constantinople_lab_to_nwb/schierek_embargo_2024/tutorials/ephys_run_session_to_nwb.ipynb

@@ -0,0 +1,205 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "352a2fb2-54fe-4418-ae34-eda86223c180",
+   "metadata": {},
+   "source": [
+    "# Neuropixels data conversion to NWB\n",
+    "\n",
+    "This notebook demonstrates how to convert an example session from the Neuropixels dataset to NWB.\n",
+    "\n",
+    "## Neuropixels dataset\n",
+    "\n",
+    "This dataset contains in vivo extracellular electrophysiology recordings from rats performing a value-based decision-making task. Neural data were acquired using Neuropixels probes (384 channels, 30 kHz sampling rate) with Neuropix-PXI hardware and OpenEphys, and preprocessed using Kilosort 2.5 with manual curation in Phy. Trials were initiated by a nose-poke in a lit center port and required maintaining a center fixation for 0.8 to 1.2 seconds, during which a tone indicated the possible reward size. A subsequent side LED indicated the potential reward location, followed by a delay period drawn from an exponential distribution (mean = 2.5 s). Rats could opt out at any time by poking the unlit port, restarting the trial. Catch trials, where the delay period only ended if the rat opted out, constituted 15-25% of the trials. Rats received penalties for premature fixation breaks. Additionally, the tasks introduced semi-observable hidden states by varying reward statistics across uncued blocks (high, low, and mixed), structured hierarchically, with blocks transitioning after 40 successfully completed trials.\n",
+    "This notebook demonstrates how to convert an example session to NWB.\n",
+    "\n",
+    "This dataset have the following data streams:\n",
+    "- Behavior: Bpod output (.mat)\n",
+    "- Recording AP, LFP: OpenEphys (binary format)\n",
+    "- Units: Phy output\n",
+    "\n",
+    "## Notes on the conversion\n",
+    "\n",
+    "The conversion notes is located in `src/constantinople_lab_to_nwb/schierek_embargo_2024/schierek_embargo_2024_notes.md`. This file contains information about the expected file structure and the conversion process.\n",
+    "\n",
+    "## Running the conversion\n",
+    "\n",
+    "To run a specific conversion, you might need to install first some conversion specific dependencies that are located in each conversion directory:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "6408c740-2030-4f51-a44b-5e6b6dece298",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "#!pip install -r ../schierek_embargo_2024_requirements.txt"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b221cfff-c7e5-44e7-bd3a-024bf2f26f67",
+   "metadata": {},
+   "source": [
+    "## Convert a single session to NWB\n",
+    "\n",
+    "The `schierek_embargo_2024_convert_session.py` script defines the `session_to_nwb` function that converts a session of Neuropixels data to NWB."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "id": "695d7f2c-5b40-4e0d-b4b7-38546a4dce52",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2024-12-04T12:39:08.133836Z",
+     "start_time": "2024-12-04T12:39:08.128903Z"
+    }
+   },
+   "source": [
+    "from constantinople_lab_to_nwb.schierek_embargo_2024.schierek_embargo_2024_convert_session import session_to_nwb\n",
+    "\n",
+    "# Parameters for conversion\n",
+    "\n",
+    "# The path to the NWB file to write.\n",
+    "nwbfile_path = \"/Volumes/T9/Constantinople/nwbfiles/sub-J076_ephys.nwb\"\n",
+    "# The OpenEphys recording folder path (make sure to include the 'Record Node #' in the folder path)\n",
+    "folder_path = \"/Volumes/T9/Constantinople/Ephys Data/J076_2023-12-12_14-52-04/Record Node 117\"\n",
+    "# The name of the *raw* recording stream (e.g. )\n",
+    "raw_stream_name = \"Neuropix-PXI-119.ProbeA-AP\"\n",
+    "# The name of the *LFP* recording stream (e.g. )\n",
+    "lfp_stream_name = \"Neuropix-PXI-119.ProbeA-LFP\"\n",
+    "# The path to the processed spike sorting file (.mat). This file contains the \"SU\" named struct that contains the processed spike data.\n",
+    "# This file also contains the \"S\" named struct containing the processed behavior data.\n",
+    "spike_sorting_mat_file_path = \"/Volumes/T9/Constantinople/Ephys Data/J076_2023-12-12.mat\"\n",
+    "# The path to the raw Bpod data\n",
+    "bpod_file_path = \"/Volumes/T9/Constantinople/raw_Bpod/J076/DataFiles/J076_RWTautowait2_20231212_145250.mat\"\n",
+    "\n",
+    "# Task specific parameters\n",
+    "\n",
+    "# The column name mapping is used to rename the columns in the processed data to more descriptive column names.\n",
+    "# New (unseen) task parameters should be added here.\n",
+    "column_name_mapping = dict(\n",
+    "    NoseInCenter=\"nose_in_center\",\n",
+    "    TrainingStage=\"training_stage\",\n",
+    "    Block=\"block_type\",\n",
+    "    BlockLengthAd=\"num_trials_in_adaptation_blocks\",\n",
+    "    BlockLengthTest=\"num_trials_in_test_blocks\",\n",
+    "    ProbCatch=\"catch_percentage\",\n",
+    "    RewardDelay=\"reward_delay\",\n",
+    "    RewardAmount=\"reward_volume_ul\",\n",
+    "    WaitForPoke=\"wait_for_center_poke\",\n",
+    "    hits=\"is_rewarded\",\n",
+    "    vios=\"is_violation\",\n",
+    "    optout=\"is_opt_out\",\n",
+    "    wait_time=\"wait_time\",\n",
+    "    wait_thresh=\"wait_time_threshold\",\n",
+    "    wait_for_cpoke=\"wait_for_center_poke\",\n",
+    "    zwait_for_cpoke=\"z_scored_wait_for_center_poke\",\n",
+    "    RewardedSide=\"rewarded_port\",\n",
+    "    Cled=\"center_poke_times\",\n",
+    "    Lled=\"left_poke_times\",\n",
+    "    Rled=\"right_poke_times\",\n",
+    "    l_opt=\"left_opt_out_times\",\n",
+    "    r_opt=\"right_opt_out_times\",\n",
+    "    ReactionTime=\"reaction_time\",\n",
+    "    slrt=\"short_latency_reaction_time\",\n",
+    "    iti=\"inter_trial_interval\",\n",
+    ")\n",
+    "# The column descriptions are used to add descriptions to the columns in the processed data.\n",
+    "# New (unseen) task parameter descriptions should be added here.\n",
+    "column_descriptions = dict(\n",
+    "    NoseInCenter=\"The time in seconds when the animal is required to maintain center port to initiate the trial (uniformly drawn from 0.8 - 1.2 seconds).\",\n",
+    "    TrainingStage=\"The stage of the training.\",\n",
+    "    Block=\"The block type (High, Low or Test). High and Low blocks are high reward (20, 40, or 80μL) or low reward (5, 10, or 20μL) blocks. Test blocks are mixed blocks.\",\n",
+    "    BlockLengthAd=\"The number of trials in each high reward (20, 40, or 80μL) or low reward (5, 10, or 20μL) blocks.\",\n",
+    "    BlockLengthTest=\"The number of trials in each mixed blocks.\",\n",
+    "    ProbCatch=\"The percentage of catch trials.\",\n",
+    "    RewardDelay=\"The delay in seconds to receive reward, drawn from exponential distribution with mean = 2.5 seconds.\",\n",
+    "    RewardAmount=\"The volume of reward in microliters.\",\n",
+    "    hits=\"Whether the subject received reward for each trial.\",\n",
+    "    vios=\"Whether the subject violated the trial by not maintaining center poke for the time required by 'nose_in_center'.\",\n",
+    "    optout=\"Whether the subject opted out for each trial.\",\n",
+    "    WaitForPoke=\"The time (s) between side port poke and center poke.\",\n",
+    "    wait_time=\"The wait time for the subject for for each trial in seconds, after removing outliers.\"\n",
+    "        \" For hit trials (when reward was delivered) the wait time is equal to the reward delay.\"\n",
+    "        \" For opt-out trials, the wait time is equal to the time waited from trial start to opting out.\",\n",
+    "    wait_for_cpoke=\"The time between side port poke and center poke in seconds, includes the time when the subject is consuming the reward.\",\n",
+    "    zwait_for_cpoke=\"The z-scored wait_for_cpoke using all trials.\",\n",
+    "    RewardedSide=\"The rewarded port (Left or Right) for each trial.\",\n",
+    "    Cled=\"The time of center port LED on/off for each trial (2 x ntrials).\",\n",
+    "    Lled=\"The time of left port LED on/off for each trial (2 x ntrials).\",\n",
+    "    Rled=\"The time of right port LED on/off for each trial (2 x ntrials).\",\n",
+    "    l_opt=\"The time of left port entered/exited for each trial (2 x ntrials).\",\n",
+    "    r_opt=\"The time of right port entered/exited for each trial (2 x ntrials).\",\n",
+    "    ReactionTime=\"The reaction time in seconds.\",\n",
+    "    slrt=\"The short-latency reaction time in seconds.\",\n",
+    "    iti=\"The time to initiate trial in seconds (the time between the end of the consummatory period and the time to initiate the next trial).\",\n",
+    "    wait_thresh=\"The threshold in seconds to remove wait-times (mean + 1*std of all cumulative wait-times).\",\n",
+    ")\n",
+    "\n",
+    "# Optional parameters\n",
+    "\n",
+    "# Whether to run a stub test conversion. The stubbed file will only contain a small portion of data.\n",
+    "# When running the full conversion stub_test should be disabled.\n",
+    "stub_test = True\n",
+    "# Whether to overwrite an existing NWB file.\n",
+    "overwrite = True\n",
+    "\n",
+    "# Run the `session_to_nwb` function with the parameters to convert to NWB\n",
+    "session_to_nwb(\n",
+    "    nwbfile_path=nwbfile_path,\n",
+    "    openephys_recording_folder_path=folder_path,\n",
+    "    ap_stream_name=raw_stream_name,\n",
+    "    lfp_stream_name=lfp_stream_name,\n",
+    "    processed_spike_sorting_file_path=spike_sorting_mat_file_path,\n",
+    "    raw_behavior_file_path=bpod_file_path,\n",
+    "    column_name_mapping=column_name_mapping,\n",
+    "    column_descriptions=column_descriptions,\n",
+    "    stub_test=stub_test,\n",
+    "    overwrite=overwrite,\n",
+    ")"
+   ],
+   "outputs": [],
+   "execution_count": 2
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "9e70b3e5-de74-4f15-bec0-688151d80ce4",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "37140fad-10d0-436d-a4a8-452083d33d10",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}