Running TXBR on SDSC Gordon Cluster

This page contains instructions on running TxBR on SDSC Gordon cluster using the TxBR singularity image.

Step 1 Building singularity image

git clone https://github.com/nbcrrolls/txbr-source
cd txbr-source
make singularity

Step 2 Copy singularity image to Gordon

• Requires
• Linux with the following installed:
• Make
• Singularity

scp dist/<TXBR img file> gordon.sdsc.edu:/oasis/projects/nsf/<ACCOUNT>/<GORDON USERNAME>

# also copy up fhv6
scp examples/data/fhv6.tar.gz /oasis/projects/nsf/<ACCOUNT>/<GORDON USERNAME>

Step 3 Start interactive job on gordon

ssh gordon.sdsc.edu
qsub -I -q normal -l nodes=1:ppn=16:native,walltime=2:00:00 -A <ACCOUNT>

Step 4 Run TxBR on fhv6

# load singularity
module load singularity/2.1.2
cd /oasis/projects/nsf/<ACCOUNT>/<GORDON USERNAME>
tar -zxf fhv6
cd fhv6
../<TXBR img file> -b fhv6a,fhv6b

Above will generate output like this:

INFO: txbr_align.py -b fhv6a,fhv6b -d /projects/ps-camera/txbr/fhv6 --wd /projects/ps-camera/txbr/fhv6 --bin 1
INFO: Load project: (b='fhv6a,fhv6b',d='/projects/ps-camera/txbr/fhv6',bin='1')
INFO: Initialize TxBR project: (/projects/ps-camera/txbr/fhv6,fhv6) ['a', 'b']
INFO: Project bin factor set to: 1
INFO: Load series: (b='fhv6a',d='/projects/ps-camera/txbr/fhv6',bin='1')
INFO: Set the bin factor of txbr.utilities.datastack.MRCStack: /projects/ps-camera/txbr/fhv6/fhv6a.preali, bin=1 to 1
.
.
.
[/projects/ps-camera/txbr/fhv6/txbr-backprojection/bin1/fhv6_z_-50.0.out] /projects/ps-camera/txbr/fhv6/txbr-backprojection/bin1/fhv6_z_-50.00.out 101
INFO: imod /projects/ps-camera/txbr/fhv6/txbr-backprojection/bin1/fhv6_z_-50.0.out /projects/ps-camera/txbr/fhv6/txbr-align/bin1/fhv6.mod
INFO: Total Time for the backprojection: 31.052032
ERROR  : Could not clear loop device /dev/loop0: (16) Device or resource busy
[churas@gcn-20-68 fhv6]$ 3dmod: cannot connect to X server 

The results will be located in this file:

txbr-backprojection/bin1/fhv6_z_-50.0.out

When done type exit to get off of compute node