Skip to content

Latest commit

 

History

History
52 lines (41 loc) · 1.57 KB

Singularity.md

File metadata and controls

52 lines (41 loc) · 1.57 KB
Raw

Singularity containers

For security reasons some environments do not allow the use of Docker containers. Instead, Singularity may be an option. Below an example of how we used Singularity on a super computing environemnt.

For more information: https://singularity.lbl.gov

# pull the docker image from Dockerhub (note: no guarantee this is up-to-date)
singularity pull docker://will5448/camera-trap-classifier:latest-gpu

# run a program
# map important paths with -B option
singularity exec --nv -B /data/my_files/:/data/my_files/ ./camera-trap-classifier-latest-gpu.simg echo "TEST"

# Alternatively switch into the singularit container and run commands there
singularity run ./camera-trap-classifier-latest-gpu.simg

# Example Usage

cd $HOME

# Prepare paths
SAVE_ROOT_PATH=$HOME
TFR_FILES=/data/tfr_files/
RUN_OUTPUT=${SAVE_ROOT_PATH}/data/run_outputs/
MODEL_SAVE=${SAVE_ROOT_PATH}/data/model_save/
LABEL_MAPPINGS=/data/tfr_files/label_mapping.json

# create output paths if they dont exist
mkdir -p $RUN_OUTPUT
mkdir -p $MODEL_SAVE

singularity exec --nv -B /data/my_files/:/data/my_files/ ./camera-trap-classifier-latest-gpu.simg \
  ctc.train \
  -train_tfr_path $TFR_FILES \
  -val_tfr_path $TFR_FILES \
  -test_tfr_path $TFR_FILES \
  -class_mapping_json  $LABEL_MAPPINGS \
  -run_outputs_dir $RUN_OUTPUT \
  -model_save_dir $MODEL_SAVE \
  -model ResNet18 \
  -labels species count standing resting moving eating interacting babies \
  -labels_loss_weights 1 0.2 0.2 0.2 0.2 0.2 0.2 0.2  \
  -batch_size 256 \
  -n_cpus 18 \
  -n_gpus 2 \
  -buffer_size 512 \
  -max_epochs 70