v0.2.2

The latest release of toffy features updated Rosetta Compensation, the deprecation of the tiling notebook, user experience improvements, along with documentation improvements and dependency housekeeping.

🚀 Features

• Check for all run merge conflicts @camisowers (#435)
• Partial FOVs clarification @camisowers (#419)
• Check for missing fovs after run is complete @camisowers (#410)
• Helper function for scaling images @camisowers (#415)

🐛 Bug Fixes

• Prevent mass column from adding during rosetta.combine_compensation_files @alex-l-kong (#444)

• Ensure low sensitivity normalization correction works with negative values too @alex-l-kong (#428)

• Use updated round 2 Rosetta folder path spec @alex-l-kong (#422)

• Space optimization for tiled Rosetta images @alex-l-kong (#421)

• Fix rosetta comparison image output @camisowers (#451)

• Add scaling to rosetta source channel images @camisowers (#440)

• Provide more robust round 1 -> round 2 copying @alex-l-kong (#426)

• Standardize indexing of over- and under-normalized channels in normalize_fov @alex-l-kong (#427)

🧰 Maintenance

• Release v0.2.2 @srivarra (#452)
• Updated the toffy environment file @srivarra (#434)
• Ensure safe conversion of panel types during modify_panel_ranges @alex-l-kong (#443)
• Update pulse height generation to be compatible with mibi-bin-tools==0.2.11 @alex-l-kong (#436)
• jupyter_contrib_nbextensions fix @srivarra (#429)

📚️ Documentation

• Indicate incompatible normalization notebook with MIBIcontrol 1.8 @alex-l-kong (#418)
• MIT License @srivarra (#448)
• Add pre-commit to standardize toffy documentation @camisowers (#437)
• Cite TSAI in tiling notebooks, deprecating them @alex-l-kong (#420)

@alex-l-kong, @camisowers and @srivarra

v0.2.1

The latest release of toffy introduces major updates to the watcher pipeline, removing duplicate data generation, re-extracting incomplete .bin files, and enabling startup without a run folder. It introduces mass proficient image extraction to address storage concerns from large .bin files, round 2 Rosetta for additional denoising, standardized curves and outlier capping for post-Rosetta normalization, and post-reorganization QC metrics to verify across TMAs. Additional features include improved tiled image stitching for TMA + tissue samples, FOV resolution QC checking, and standardized output folders on the D drive for MIBI computers.

🚀 Features

• Check for partially generated fovs @camisowers (#416)
• Remove hanging extracted_imgs_dir reference in Rosetta notebook @alex-l-kong (#413)
• QC Metrics - Save Longitudinal Transformed Metrics @srivarra (#408)
• Ensure watcher can wait on run folder to appear @alex-l-kong (#401)
• QC for TMAs and Control Samples @srivarra (#358)
• FOV resolution check @camisowers (#391)
• Adjust rosetta processing paths @camisowers (#385)
• Tma and tiled run stitching @camisowers (#382)
• Mass proficient extraction for NAS storage @alex-l-kong (#342)
• Add Rosetta V2 processing functionality @alex-l-kong (#319)
• Ensure runs below fovs_per_num threshold get skipped during Rosetta t… @alex-l-kong (#406)
• Skip watcher callbacks for FOVs that already have data generated @alex-l-kong (#394)
• Ensure incompletely extracted bin files get re-extracted @alex-l-kong (#392)
• Turn off combined.csv file warnings for intermediate callbacks in watcher @alex-l-kong (#393)
• Turn on y ticks for QC plots @alex-l-kong (#386)
• Norm workflow @alex-l-kong (#361)
• Cap normalization at 10X upper bound, and 1X lower bound @alex-l-kong (#352)
• Add functionality to catch file renaming to watcher @alex-l-kong (#359)
• Remove warning for .def.hist and .pro.hist files @alex-l-kong (#381)
• Allow for image specified for flatfield correction to be empty @cliu72 (#383)

🐛 Bug Fixes

• Fix rescale_raw_images error @camisowers (#398)
• Ensure pulse heights file is named with _ not - @alex-l-kong (#412)
• Make sure watcher logging actually writes to file on CACs @alex-l-kong (#390)
• Add a scale factor for image_stitching.stitch_images for Photoshop compatibility @alex-l-kong (#374)

🧰 Maintenance

• Update environment.yml @srivarra (#372)
  @alex-l-kong, @camisowers, @cliu72 and @srivarra

v0.2.0

The latest release of toffy includes some bug fixes to median pulse heights, normalization for all-zero pulse height channels and intermediate callback plotting per FOV, as well as updating multiple tile stitching to be more flexible. In addition a few small bugs were fixed such as missing imports, and updating documentation for installing pre-commit. Our GitHub Actions pipeline has also received a light cleanup.

🚀 Features

• GitHub Actions - CI Updates @srivarra (#370)
• Multiple tile stitching @camisowers (#360)

🐛 Bug Fixes

• Refresh intermediate callback plotting on each new FOV @alex-l-kong (#363)
• Fix median pulse height chart drawn by visualize_mph @alex-l-kong (#356)
• Added conda-forge channel to environment.yml @srivarra (#354)
• Ensure normalization doesn't happen for all-zero pulse height channels @alex-l-kong (#326)
• Update axes_size to axes_font_size in 3c notebook @alex-l-kong (#343)
• Missing import in extraction notebook @camisowers (#334)

🧰 Maintenance

• Next Release - v0.1.3 @srivarra (#371)

📚️ Documentation

• Fix brew install command for pre-commit @alex-l-kong (#341)
• Update README.md @ngreenwald (#333)
  @alex-l-kong, @camisowers, @ngreenwald and @srivarra

v0.1.2

Adjusts toffy to use the new alpineer repo containing shared utility functions between toffy and ark-analysis. Also adds in a few new features such as tiled image stitching and real time plotting for the watcher notebook. Switches from Travis-CI to GitHub Actions.

🚀 Features

• GitHub Actions @srivarra (#328)
• Tiled stitching for toffy @camisowers (#285)
• Flexible Requirements @srivarra (#330)
• Add ability to override run callbacks to process per FOV @alex-l-kong (#313)
• Update Rosetta test directory setup @alex-l-kong (#300)

🐛 Bug Fixes

• Skip empty json files in voltage check @camisowers (#290)
• Don't display warning if the watcher discovers a temp file @alex-l-kong (#329)
• Remove MIBItracker tests @alex-l-kong (#323)
• Make sure Rosetta converts images to float prior to running @alex-l-kong (#293)

🧰 Maintenance

• Conda Environment - scikit-learn & seaborn @srivarra (#297)
• Substituted Ark imports with TMI in the Jupyter Notebooks @srivarra (#295)
• Adding TMI @srivarra (#286)
• add tmi to environment.yml @camisowers (#284)
• Image Utils in TMI @srivarra (#282)
  @alex-l-kong, @camisowers and @srivarra

v0.1.1

Introduces small bug fixes for loading in panels, extraction crash logic improvements with json files and lossless compression for tiffs on by default.

🚀 Features

• Compressed Tiff Files @srivarra (#280)

🐛 Bug Fixes

• Make sure loading in convert_panel happens in utf-8 @alex-l-kong (#271)
• Fix extraction logic for run with a single fov crash @camisowers (#269)

🧰 Maintenance

• Next Release - v0.1.1 @srivarra (#281)

📚️ Documentation

• Add link in notebooks for the ark tiled Image Stitching script @camisowers (#278)
• Toffy Setup Video Fix @srivarra (#267)
  @alex-l-kong, @camisowers and @srivarra

v0.1.0

First release of the beta version of toffy. This includes new MIBI image generation and processing tools, organized into a multistep pipeline for users to follow. Also contains adjustments made in preparation for and following the MIBI Workshop.

🚀 Features

• Notebook 1 normalization checks @camisowers (#250)
• Add _pulse_height.csv file generation to watcher @camisowers (#262)
• Custom error for Rosetta file copying @camisowers (#258)
• Json file encoding @camisowers (#243)
• Stitch images function without run file @camisowers (#237)
• Toffy notebook fixes @camisowers (#230)
• Create release-drafter.yml @srivarra (#223)
• Rosetta stitching fix @camisowers (#227)
• Add interactive tiled region overlay @alex-l-kong (#86)
• Add temporary scripts directory to track notebook updates @alex-l-kong (#220)
• Rosetta notebook testing updates @camisowers (#212)
• Optimize Watcher Tests @ackagel (#219)
• Remove moly_path requirement for remap_to_reorder_fovs @alex-l-kong (#215)
• Update run metrics saving behavior @camisowers (#213)
• Add checks for out-of-bounds coordinates during tiling process @alex-l-kong (#209)
• Panel conversion implementation @camisowers (#202)
• Added .DS_Store in the .gitignore @srivarra (#206)
• qc metrics violin plots on normal tissues @mheskett (#153)
• Ensures co-registration path is absolute @alex-l-kong (#196)
• Image stitching in watcher @camisowers (#152)
• Adjust min TMA size to 2x2 @alex-l-kong (#194)
• Simplify QC metric visualization procedure @alex-l-kong (#174)
• Output float32 images for normalization notebook @camisowers (#155)
• run fovs in numerical order @camisowers (#154)
• Check voltage @camisowers (#148)
• Split run file @camisowers (#145)
• Update QC notebook with real data @camisowers (#147)
• Extract bin notebook @camisowers (#130)
• Mph plots @camisowers (#101)
• Mph histograms @camisowers (#126)
• Implement json helper funcs @mheskett (#142)
• Json helper functions @mheskett (#138)
• Small tweaks @ngreenwald (#137)
• Format data for analysis @ngreenwald (#131)
• added moly_path option @srivarra (#114)
• Mph norm workflow @ngreenwald (#125)
• find moly FOVs based on JSON file @ngreenwald (#124)
• Rosetta simplification @ngreenwald (#107)
• implements watcher chaining architecture @ackagel (#99)
• Rosetta modifications @ngreenwald (#92)
• Rename warning @camisowers (#103)
• Update repo setup for watcher @ngreenwald (#93)
• Rename fov folders @camisowers (#85)
• Streak Detection @srivarra (#71)
• adds fov_watcher template @ackagel (#53)
• Don't check contrast for skimage save function. @srivarra (#58)
• Add ROI name prefix to tiled region FOV names @alex-l-kong (#66)
• Rosetta workflow @ngreenwald (#43)
• Write co-registration settings to JSON param file @alex-l-kong (#60)
• Add watcher compatible QC notebook @alex-l-kong (#50)
• Add notebook to change co-registration parameters in settings.py @alex-l-kong (#45)
• Add tiled regions notebook to toffy @alex-l-kong (#41)
• Normalize images by MPH @ngreenwald (#35)
• Change x to columns and y to rows in TMA script @alex-l-kong (#39)
• Add TMA FOV validation functions for user sanity checking @alex-l-kong (#30)
• Add TMA tiling utils into creed @alex-l-kong (#13)
• Support non-rectangular TMAs @ngreenwald (#19)
• Automatic checking of hashes for transferred files @ngreenwald (#21)
• Add automatic detector sweep ID @ngreenwald (#17)
• Add Rosetta matrix functionality @ngreenwald (#27)
• Add coveralls support @alex-l-kong (#14)
• Add Travis and pytest support @alex-l-kong (#15)
• Add notebooks @ackagel (#3)

🐛 Bug Fixes

• get_max_img_size works with subdir @camisowers (#264)
• Rosetta mass 39 naming conflict @camisowers (#252)
• Panel conversion column typo @camisowers (#247)
• Ignore stitched_images folder in rosetta processing @camisowers (#239)
• Ensure path to rosetta_matrices is added in the setup notebook @alex-l-kong (#236)
• Catches tempdir cleanup error @camisowers (#225)
• Make watcher skip moly points @ackagel (#117)
• Remove endline in column names from correctly formatted panel @camisowers (#211)
• Fix pulse_height_dir path in normalize notebook @alex-l-kong (#210)
• Watcher hidden files @camisowers (#193)
• Added the location of the Optical Image File in the Notebook @srivarra (#182)
• Empty json check @camisowers (#187)
• Make sure mapping_path load call gets assigned to mapping prior to relabeling TMA FOV names @alex-l-kong (#186)
• Adjusted Streak Detection mask generation parameters @srivarra (#119)
• Fix import error @camisowers (#96)
• Add scheme for renaming TMA manual FOV duplicates @alex-l-kong (#84)
• Ensure JSON data is read in with encoding UTF-8 @alex-l-kong (#36)
• Change notebook import from creed to toffy @alex-l-kong (#33)
• Update repo for toffy @ngreenwald (#32)

🧰 Maintenance

• Adjusted requirements to work with ark-analysis on Python 3.7 @srivarra (#132)
• Pin ark requirement @ngreenwald (#121)
• Updating the License @srivarra (#47)

📚️ Documentation

• Add workshop links to readme @camisowers (#260)
• Update flowchart in readme @camisowers (#248)
• Update README.md with link to discussions @ngreenwald (#229)
• Reorg notebook cleaning @camisowers (#222)
• Add documentation for TMA notebook @alex-l-kong (#207)
• Update readme with directory structure @camisowers (#190)
• Added Step 3e to the pipeline flowchart @srivarra (#204)
• Notebook 4a - Documentation Improvements @srivarra (#177)
• Toffy Pipeline Flowchart @srivarra ([#200](https://github....