v1.2.1

Conda users must update pipeline's Conda environment

$ bash conda/update_conda_env.sh

• bug fixes

  • fix for new conda (conda activate)

• others

  • fix kentutil version at v377 (docker/singularity/conda)

v1.2.0

utils/resumer/resumer.py has been depcreated. Also, all tutorials and old documentation are abandoned. Please use Caper to run our pipeline. To organize outputs from metadata.json (output of Cromwell), use Croo

$ pip3 install caper croo

• Bug fixes

  • Remove intermediate files gradually as macs2 task progresses
  • Increase default disk size for Google Cloud for some heavy tasks (macs2, spp, bwa, filter, fingerprint)
  • new conda (conda activate instead of source activate)

• Others

  • Upgrade cromwell.jar from 34 to 38
  • split macs2 task into macs2 and macs2_signal_track
    • change in output name for signal track
      • macs2.sig_pval -> macs2_signal_track.pval_bw
      • macs2.sig_fc -> macs2_signal_track.fc_bw
    • task level test for two tasks
  • 3-dim fastqs array is not supported anymore
  • exclusion range for cross-correlation analysis
    • adding -x=min:max to run_spp.R in xcor
    • this will prevent xcor estimates wrong (negative) fragment len
  • fix KentUtils version at v369 in docker/singularity containers
  • added new example for Caper

v1.1.7

• bug fixes
  • catch exit code correctly in a bash pipe
  • glob idr_log corretly
  • fix broken URIs for DX in documentation
  • fix WDL parse error
  • fix for missing matplotlib module error in conda env
  • removed ENTRYPOINT from Dockerfile for AWS backend
• others
  • added bwa mem for PE dataset (experimental)
  • added count signal track
  • sync with atac, chip-nexus
  • added parameter chip.use_filt_pe_ta_for_xcor
  • added support for .fastq file
• update docs
  • better description for outputs in output.md
  • update for SCG (using moduled miniconda/3)

Conda users must update pipeline's Conda environments.

$ bash conda/update_conda_env.sh

v1.1.6.1

Minor release

• Conda users need to update conda environment with the following command line:

$ bash conda/update_conda_env.sh

• bug fixes
  • update incorrectly-built conda package.
  • workaround for cromwell-38 wom API bug (broadinstitute/cromwell#4659)
  • fix for singularity backend hanging problem. use built-in background processing in cromwell instead of & echo $! to print PID.
  • allow irregular chr names in *.hammock.gz (used for Wash U genome browser).
  • fix grep bug in encode_trim_fastq.py

    Traceback (most recent call last):
      File "/software/chip-seq-pipeline/src/encode_trim_fastq.py", line 64, in <module>
        main()
      File "/software/chip-seq-pipeline/src/encode_trim_fastq.py", line 55, in main
        trimmed = trim_fastq(args.fastq, args.trim_bp, args.out_dir)
      File "/software/chip-seq-pipeline/src/encode_trim_fastq.py", line 42, in trim_fastq
        if int(run_shell_cmd(cmd2))>0:
      File "/software/chip-seq-pipeline/src/encode_common.py", line 252, in run_shell_cmd
        raise Exception(err_str)
    Exception: PID=18, PGID=18, RC=1
    STDERR=
    STDOUT=0
    

v1.1.6

• how to update pipeline (Conda users only)

  • run git pull to get updates from master branch (v1.1.6).
  • run bash conda/update_conda_env.sh to update pipeline's Conda environment.

• can resume a failed workflow from where it left off

  • utils/resumer/resumer.py: make a new input JSON file from a failed workflow

• bug fixes

  • remove raw peak stats from ATAQC and add idr/overlap peak stats (issue #74)
  • fix subsampled output filenaming (..*M. -> .*M.)
  • fix macs2_pr1/pr2 error when using chip.true_rep_only
  • fix error in trimfastq when read length is < chip.trim_bp

• others

  • update docs for better user experience

v1.1.5

How to update pipeline for v1.1.5

• run git pull to get updates from master branch (v1.1.5)

• Conda users only

  • run bash conda/update_conda_env.sh to update Conda environments with new source codes (v1.1.5)

• qc.json parser

  • gather quality metrics in multiple qc.json into a single TSV file

• bug fixes

  • remove overlapping regions in bedgraph in macs2
  • Conda installation destination absolute path length limit: 80->255

• others

  • shared singularity image on sherlock/scg (users don't need to build one)
  • update docs (new easier tutorial for sherlock/scg/slurm/sge)
  • support for Windows 10

v1.1.4

• new user-friendly documentation
• bug fixes
  • workaround for non-zero exit code not caught by cromwell
  • possible fix for UTF8 locale issue in fingerprint task
• others
  • CircleCI status badge

v1.1.3

• bug fixes
  • case sensitive sorting (LC_COLLATE=C) for UCSC tools
  • workaround for cromwell/WDL stdout() issue
  • fix for SPP crash (when nth==1)
  • picard SEGFAULT
• others
  • support for PBS cluster
  • fix GC thread=1 for all java apps

v1.1.2

• code refactoring for better UI
  • all task level parameter variables (including resource settings) are moved to a workflow level. so that we can use the same input JSON format for dxWDL and cromwell. it's easier for users not to have task-level variables. it was very confusing.
• support for custom-built genomes with irregular chromosome name.
  • added keep_irregular_chr_in_bfilt_peak to keep irregular (which does not match pattern chr[\dMXY]+) chromosome name in peaks.
  • disabled chromosome name check in bedtools intersect (added -nonamecheck).
• bug fixes
  • fixed bug in xcor (when subsample is explicitly given as 0).
  • fixed typos in docs.
• CircleCI
  • google auth for workflow test (no need to submit jobs to cromwell server)
  • good bye Jenkins
• others
  • added genome data downloader (dowloading from GC).
  • added temporary directories in docker container (/mnt/ext_*/).
  • updated docs for singularity 3.0 (due to some changes in singularity pull)

v1.1.1

• fix for no-read case (toy problem)
• fix python3 base64 encode/decode error in qc_report
• improved qc_report (added help, change of chapter order, chapter summary, ataqc), qc.json
• hammock generation for Wash U browser track
• fixed docker cache/Jenkins problem (rebuilt from scratch, removed COPY)
• better debugging (printing task's CMD, STDOUT)
• fixed R package problem (fixed broken CRAN repo, prevent conflict between Conda and local Rs)
• fixed all singularity problems (hanging, error logging)
• new genome data/test samples location for SCG/Sherlock