canu assembler

HOME | Biocontainer Compatibility Report | Package Installation Benchmark | GPGPU Access | Containerize Canu Assembler | Composability Feature | Developer Notice | Incompatible Images

Containerize Canu Assembler

The cans assembler can not be containerized due to the fact that all modern containers do not have the composability feature, i.e. allowing the canu assembler to directly call qsub / qhost / qstat and etc commands on the host, the reason why modern container systems do not support such feature is that they all focus on providing isolation feature and dependency hell issue is not the primary problem they aiming at solving.

LPMX solves this issue in novelty via its composability feature, as it is extremely easy for LPMX to replace executable in the air. To containerize the canu assembler and make it fully runnable inside LPMX, e.g. allowing the canu assembler to generate script and calling qsub (Use SGE as the example) to submit this script for job dispatch, following several steps are needed:

Important Notice: only SGE is supported currently, please check the update from this repository: https://github.com/JasonYangShadow/fakebatchtool

1. Create a container including the canu assembler via LPMX option "-e, --engine"

   docker create sub-command is the advanced command of lpmx, which is used for initializing and running the images downloaded from docker hub
   
   Usage:
     lpmx docker create [flags]
   
   Flags:
     -e, --engine string   use engine(optional)
     -h, --help            help for create
     -n, --name string     optional
     -v, --volume string   optional
    
   example:
   	lpmx docker create -n container_name -e SGE ubuntu:18.04
   	
   this command will create a container based on ubuntu:18.04 Docker image and enable SGE related environment variables inside containers, e.g. SGE_CELL, SGE_CLUSTER_NAME, SGE_ROOT, which are required by the canu assembler.
   

   download latest canu assembler release from its Github repository

   extract the package and install necessary dependencies, e.g. default-jre, perl

   enable loading libraries from the host inside containers

   export FAKECHROOT_USE_SYS_LIB=true
   

   then test if canu assembler works inside containers

   

2. To make sure qsub, qhost and other SGE commands available inside containers

   set the executable mappings without stopping containers

   fakeqsub is available in this repository: https://github.com/JasonYangShadow/fakebatchtool

   Usage:
     lpmx set [flags]
   
   Flags:
     -c, --filecache      use FileCache rather than Memcached(optional)
     -h, --help           help for set
     -i, --id string      required(container id, you can get the id by command 'lpmx list')
     -n, --name string    required(should be the name of libc 'system calls wrapper' or mapped program path)
     -t, --type string    required('add_map','remove_map','add_exec', 'remove_exec')
     -v, --value string   required in add mode(value(file1:replace_file1;file2:repalce_file2;) or a mapped path) while optional in remove mode
     
   example:
   lpmx set -i container_id -t add_exec -n /bin/qsub -v /home/xxx/fakeqsub -c
   lpmx set -i contaienr_id -t add_exec -n /bin/qhost -v /home/xxx/qhost -c
   

   users can also check executable mapping results

   Usage:
     lpmx get [flags]
   
   Flags:
     -c, --filecache     use FileCaache rather than Memcached(optional)
     -h, --help          help for get
     -i, --id string     required
     -n, --name string   required
     
   example:
   lpmx get -i container_id -n /bin/qsub -c
   

   previous settings will be returned:

   

   then to enable executable mapping in the air

   export FAKECHROOT_EXEC_SWITCH=true
   

   then exposed commands should be visible inside containers as shown in the following figure

   

3. Run canu assembler as usual, as qsub is trapped inside containers and replaced by the fakeqsub, the fakeqsub will generate a wrapping script when qsub is called via the canu assembler ,which will be dispatched to other computing node for further steps.