templates/3a_monitor_MIBI_run.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Monitoring an ongoing MIBI run\n",
    "\n",
    "This notebook can be run alongside an active MIBIScope run.  As images are generated, this notebook will automatically pass the data through pre-specified functions, like tiff \n",
    "extraction, qc metric computation, etc. Eventually, all of the processing steps in the toffy repo will be incorporated into the Watcher functionality here. For now, this notebook will automatically extract tiffs, compute pulse height data, stitch images by channel, and generate the QC and MPH plots of your data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "from toffy.panel_utils import load_panel\n",
    "from toffy.fov_watcher import start_watcher\n",
    "from toffy.watcher_callbacks import build_callbacks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Required variables\n",
    "\n",
    " - `run_name` should contain the exact name of the MIBI run that will be monitored\n",
    " - `panel_path` should point to a panel csv specifying the targets on your panel. You can download your panel online from the Ionpath MibiTracker and then copy the file to the `C:\\\\Users\\\\Customer.ION\\\\Documents\\panel_files` directory (see [panel format](https://github.com/angelolab/toffy#panel-format) for more information)\n",
    "  - `extract_prof` indicates whether you want to include mass proficient extraction on top of the default (mass deficient) extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the name of the run\n",
    "run_name = 'YYYY-MM-DD_run_name'\n",
    "\n",
    "# path to user panel\n",
    "panel_path = 'C:\\\\Users\\\\Customer.ION\\\\Documents\\\\panel_files\\\\my_cool_panel.csv'\n",
    "\n",
    "# whether to include proficient extraction or not\n",
    "extract_prof = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Everything necessary for and subsequently outputted from this notebook is stored in the automatic directories established in `1_set_up_toffy.ipynb`. More information on the uses and locations of the directories in toffy can be found in the [README](https://github.com/angelolab/toffy#directory-structure)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load panel\n",
    "panel = load_panel(panel_path)\n",
    "\n",
    "# these are set automatically\n",
    "base_dir = os.path.join('D:\\\\Data', run_name)\n",
    "extraction_dir = os.path.join('D:\\\\Extracted_Images', run_name)\n",
    "metrics_data_dir = os.path.join('C:\\\\Users\\\\Customer.ION\\\\Documents\\\\run_metrics', run_name, 'fov_data')\n",
    "metrics_plot_dir = os.path.join('C:\\\\Users\\\\Customer.ION\\\\Documents\\\\run_metrics', run_name)\n",
    "\n",
    "# path to log folder\n",
    "log_path = os.path.join('C:\\\\Users\\\\Customer.ION\\\\Documents\\\\watcher_logs', run_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Callback initialization\n",
    "\n",
    "Callbacks listed in `fov_callbacks` arugment will be run on each completed FOV, the moment both the .bin & .json are found.\n",
    "\n",
    "* The `extract_tiffs` callback specifies that every FOV generates tiffs according to the supplied `panel`. <br/> (See [3b_extract_images_from_bin](./3b_extract_images_from_bin.ipynb) for more details.)\n",
    "\n",
    "* The `generate_pulse_heights` FOV callback computes the median pulse heights for each mass specified in the `panel`. \n",
    "<br/> (See [4b_normalize_image_data](./4b_normalize_image_data.ipynb) for more details.)\n",
    "\n",
    "Callbacks listed in the `intermediate_callbacks` argument run and update as each new FOV is generated.\n",
    "\n",
    "* The `plot_qc_metrics` intermediate callback will run all currently available qc metrics on each FOV and plot the results. <br/> (See [3c_generate_qc_metrics](./3c_generate_qc_metrics.ipynb) for more details.)\n",
    "\n",
    "* The `plot_mph_metrics` intermediate callback will compute the median pulse height data for each  FOV and plot the results. Additional arguments are: `regression` which when set to True will also plot the linear regression line for the data. <br/> (See [3d_compute_median_pulse_height](./3d_compute_median_pulse_height.ipynb) for more details.)\n",
    "\n",
    "Callbacks listed in the `run_callbacks` argument will be run only once all expected FOV's have been discovered and processed. \n",
    "\n",
    "* The `image_stitching` run callback will create a single image, which stitched together all FOV images for a specific channel. Additional arguments are: `channels`. <br/> (See [3e_stitch_images](./3e_stitch_images.ipynb) for more details.)\n",
    "\n",
    "* The `check_incomplete_fovs` run callback will check the run for any partially generated images. <br/> (See [3b_extract_images_from_bin](./3b_extract_images_from_bin.ipynb) for more details.)\n",
    "  \n",
    "* The `check_missing_fovs` run callback checks that the run produces the appropriate .bin and .json all files for all FOVs included in the run file. <br/> (See [3f_FOV_checks](./3f_FOV_checks.ipynb) for more details.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fov_callback, run_callback, intermediate_callback = build_callbacks(\n",
    "    run_callbacks = ['image_stitching', 'check_incomplete_fovs', 'check_missing_fovs'],\n",
    "    intermediate_callbacks = ['plot_qc_metrics', 'plot_mph_metrics'],\n",
    "    fov_callbacks = ['extract_tiffs', 'generate_pulse_heights'],\n",
    "    extract_prof=extract_prof,\n",
    "    tiff_out_dir=extraction_dir,\n",
    "    qc_out_dir=metrics_data_dir,\n",
    "    mph_out_dir=metrics_data_dir,\n",
    "    pulse_out_dir=metrics_data_dir,\n",
    "    plot_dir=metrics_plot_dir,\n",
    "    save_dir=metrics_plot_dir,\n",
    "    panel=panel,\n",
    "    warn_overwrite=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "start_watcher(base_dir, log_path, fov_callback, run_callback, intermediate_callback)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "vscode": {
   "interpreter": {
    "hash": "f6f196d8c15ef51fc3b92e6b5462f515514162e36d9ad988b37a3636f71a6ec3"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}