Cloud DeepCell - Scaling Image Analysis

This working Repo contains our notes / utilities / info for our cloud DeepCell imaging project.

Here is the high level workflow for using DeepCell:

_{lucidchart source}

Note that DeepCell itself does not process TIFF files. The TIFF channels must be extracted into Numpy arrays first.

Also note that DeepCell performs its own pre- and post-processing around the TensorFlow prediction. In particular, DeepCell divides the input into 512x512 tiles which it predicts in batches, then reconstructs the overall image.

Goal and Key Links

• GOAL: Understand and optimize using DeepCell to perform cellular image segmentation on GCP at scale.
  • KEY LINK #1: our benchmarking process.
  • KEY LINK #2: our support/testing notebooks.
  • KEY LINK #3: our project board & work areas for this project.

Findings

GPU makes a dramatic difference in model inference time.

Memory usage increases linearly with number of pixels.

Optimization opportunities

Here are some areas we've identified:

• Preprocessing
  • DeepCell converts everything to 64bit float. That's memory intensive. Do we actually need to?
• Prediction
  • Benchmark results (GPU, batch size, resolution)
  • Investigate Google TensorFlow optimizations
• Postprocessing
  • h_maxima: need to ship a ~15x speedup optimization

Local development

Mac OS x86_64

Nothing special. You just need Python 3.10 at the latest.

python3.10 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Mac OS arm64

Some incantations are needed to work on Apple silicon computers. You also need Python 3.9.

DeepCell depends on tensorflow, not tensorflow-macos. Unfortunately we need tensorflow-macos specifically to provide TF2.8 on arm64 chips.

The solution is to install the packages one at a time so that the DeepCell failure doesn't impact the other packages.

python3.9 -m venv venv
source venv/bin/activate
pip install -r requirements-mac-arm64.txt
cat requirements.txt | xargs -n 1 pip install

# Let it fail to install DeepCell, then:
pip install -r requirements.txt --no-deps

# Lastly install our own library. Note --no-deps
pip install --editable . --no-deps

I think but am not sure that the first --no-deps invocation is unnecessary as pip install installs dependencies.