3.2.0

What's Changed

Important CRAM writing bug fix

A serious bug which can cause corrupted reads in CRAM files was discovered and fixed in this release. This bug was introduced in HTSJDK 3.0.0, and affects Picard versions 2.27.3 through 3.1.1 and GATK versions 4.3 through 4.5.

The bug occurs in cases where there is a read aligned starting at exactly position 1 on a reference contig. This means that the bug doesn't generally impact human autosome and X/Y contigs because they tend to start with a large number of N bases and reads are not aligned at exactly position 1. The exceptions to this would be T2T references and things like mitochondrial calling.

For more information on the conditions that trigger this bug, see this post.

GATK 4.6 includes a tool called CRAMIssue8768Detector that can scan a CRAM file and report whether it is affected, and if so which regions in the file are corrupt. If you suspect that some of your CRAM files may have been affected, please run this tool on them for confirmation!

See also samtools/htsjdk#1708 for more information.

Better support for remote files

Improvements to allow direct access to remote files continue. It's now possible to use a remote reference file without localizing it in many cases. Files which are available through http URLs are now accessible directly as well. (ex: https://example.com/my.bam). This allows direct access to signed URLs, although index and supporting files may not be discoverable automatically.

• Enable use of cloud reference files by @cmnbroad in #1804
• Adding http-nio as a dependency by @lbergelson in #1929

New features for flow based reads

• MarkDuplicates strategy of flow based reads that looks only at the qualities close to the end of the read by @ilyasoifer in #1942
• CollectQualityYieldMetricsFlowSpace tool by @dror27 in #1932

New Options

• KEEP_ZERO_LENGTH_INTERVALS flag when converting bed -> interval_list by @rickymagner in #1928
• Make the VCF option in CollectSamErrorMetrics optional. by @nh13 in #1476
• Add the EXT argument to CollectSamErrorMetrics. by @nh13 in #1478

Bug fixes

Bug fixes to several tools as well as important CRAM fixes from an updated htsjdk

• MarkDuplicates: Add read group ID instead of string "RG" by @michaelgatzen in #1937
• Fix CollectHSMetrics - Don't use Coverage Cap on fold score by @JoeVieira in #1913
• Fix for order flipping in SortingCollection used for MarkDuplicates by @wook-choi in #1945
• Allow fingerprinting of SAM files that only have a partial dictionary match to the haplotype map by @yfarjoun in #1955
• Fix a bug in the liftover logic by @yfarjoun in #1956

Documentation and improved error messages

• Reject piped input (/dev/stdin) for BedToIntervalList by @kockan in #1918
• Update AbstractAlignmentMerger.java Warning Message for Cross Species Contamination by @gokalpcelik in #1960
• MergeBamAlignment documentation by @kachulis in #1922
• Updated SamToFastq documentation by @kockan in #1920

Maintenance and dependency updates

• Update setup_gcloud github action by @lbergelson in #1939
• Update Dockerfile and build_push_docker.sh by @lbergelson in #1921
• Update Gradle to 8.5 by @lbergelson in #1930
• Convert to use the java-library plugin by @lbergelson in #1934
• update htsjdk to 4.1.1 by @lbergelson in #1967
• Update http-nio to 1.1.1 by @droazen in #1968

New Contributors

• @JoeVieira made their first contribution in #1913
• @wook-choi made their first contribution in #1945
• @dror27 made their first contribution in #1932
• @gokalpcelik made their first contribution in #1960

Full Changelog: 3.1.1...3.2.0

3.1.1

This is a minor release with a bug fixes and a dependency update.

What's Changed

• Check for non-symbolic alt alleles before left aligning in liftover by @kachulis in #1909
• Fix for setting the custom tmp folder. by @kockan in #1908
• FastqToSam stdin fix for #915 by @delocalizer in #1910
• Changed CrosscheckFingerprints hashing to linked hashing by @tmelman in #1892
• Update htsjdk to 4.0.2 by @lbergelson in #1917

New Contributors

• @delocalizer made their first contribution in #1910

Full Changelog: 3.1.0...3.1.1

3.1.0

Cloudy Edition

Highlights

Improving Google Cloud Support

We've done away with the picard-cloud.jar. The default picard jar now includes the necessary libraries to read from google cloud buckets. This is part of an ongoing push to cloud enable more of the important picard tools. It will reduce confusion and simplify code.

• cloud jar is now default jar #1900
• IntervalListTools is cloud enabled #1852
• updated google nio library which supports _ in bucket names #1903
• new test infrastructure for writing tests that use google cloud #1860

IntelGKL update

A new version

• A new version of the Intel GKL library fixes several major bugs. #1901
• A bug fix which enables the IntelInflater which was accidentally being ignored. #1885

Dependency and Gradle Updates

• Updated dependencies to their latest versions to incorporate security and bug fixes.
• Updated to gradle 8.2.1

Notes

• Downstream toolkits will have to make some changes if they want to continue to NOT provide the google cloud libraries.
• Updating to gradle 8.2.1 was necessitated by a packaging issue. One of the updated dependencies now includes java 19 code in it's multi-version jar and we needed to update to the newest version of the shadowJar plugin in order to handle it. See #1903 for more disussin.

Change List

• Remove unused method Fingerprint.filterSuspectSites() by @lbergelson in #1849
• Fix crash on PF_READS == 0 for AlignmentSummaryMetricsCollector by @kockan in #1871
• More informative error messages when dictionaries don't match by @kachulis in #1870
• Enable IntelInflater by @lbergelson in #1885
• Add .DS_STORE and .java-version to .gitignore by @lbergelson in #1886
• Fix deprecated uses of newInstance() by @lbergelson in #1887
• Add tests to verify sequence dictionary MD5s respect dos line endings. by @cmnbroad in #1884
• Mark GENOTYPING_ERROR_RATE as a DeprecatedFeature and remove code ref… by @cmnbroad in #1878
• Refactoring by @LadDeep in #1868
• Fix #1081. Make --USE_SEQUENTIAL_FASTQS help message more user friendly by @nrockweiler in #1891
• DbSnpBitSetUtil should verify that the VCF dict should match the inpu… by @nh13 in #1889
• Fix the dependency issue causing dsp-appsec-trivy workflow to fail. by @kockan in #1895
• Lb add cloud tests by @lbergelson in #1860
• make cloud jar mandatory by @lbergelson in #1900
• Support cloud input/output for IntervalListTools by @takutosato in #1852
• typo correction "Didn't recognized" -> "Didn't recognize" by @tomkinsc in #1902
• Update google-cloud-nio to support underscores in bucket names by @kshakir in #1903
• Fix gradle 8 publishing issue by @kshakir in #1904
• Update all our dependencies to the latest version by @lbergelson in #1901

New Contributors

• @kockan made their first contribution in #1871
• @LadDeep made their first contribution in #1868
• @nrockweiler made their first contribution in #1891
• @tomkinsc made their first contribution in #1902
• @kshakir made their first contribution in #1903

Full Changelog: 3.0.0...3.1.0

3.0.0

Picard 3.0.0:

The same as before but now on Java 17

The big change is that Picard is moving from Java 8 -> 17.
Java 8 is very old and no longer available by default on most systems. Java 17 is the newest long term support release. This will enable us to take advantage of improvements in the language as well as benefiting from improvements and security fixes in the newer JVMs.

Major changes

• Upgrade to Java 17. by @cmnbroad in #1833

Bug fixes and minor improvements

• Improvements to CompareMetrics by @kachulis in #1839
• Fix typo in a filter name output by LiftoverVcf by @lbergelson in #1842
• Stop removing periods from the end of output names by @tlangs in #1433
• Avoid merging fingerprints when there is only one by @yfarjoun in #1826
• Remove commons.lang dependency because it is redundant with commons lang3 by @lbergelson in #1856
• Delete the now redundant ClassFinder by @lbergelson in #1857
• Remove obsolete references to Google Genomics API by @lbergelson in #1858
• Fix Duplicate Set Index for queryname sorted input to MarkDuplicates by @kachulis in #1843
• Allow empty input files for CollectAlignmentSummaryMetrics and FastqToSam by @kachulis in #1859

Documentation and infrastructure changes

• Update CrosscheckFingerprints summary by @derekca in #1721
• Updated the issue template to link to our support forum by @droazen in #1837
• Adding a gh actions test by @lbergelson in #1853

Full Changelog: 2.27.5...3.0.0

2.27.5

What's Changed

• Allow explicit index file paths for inputs to CrosscheckFingerprints by @rickymagner in #1823
• Fix temporary directory hijacking or temporary directory information disclosure by @lbergelson in #1829
• Update htsjdk to 3.0.1 by @lbergelson in #1831

Full Changelog: 2.27.4...2.27.5

2.27.4

• Picard modifications to support flow based sequencing (#1813)

2.27.3

• Upgrade to htsjdk 3.0.0. (#1812)
• Fix FastqToSam regression. (#1811)

2.27.2

• Update command line parser argument style detection, add tests. (#1785)
• Convert FastqToSam and RevertSam to use PicardHtsPath (#1794)

2.27.1

• Add a new interval list scatter mode (#1786)

2.27.0

• Upgrade google-cloud-nio to match gatk (#1792)
• Add support for documentation generation for metrics classes. (#1782)
• Simplify handling of paths for input in BuildBamIndex.java (#1787)