Skip to content

How to release Picard

Jump to bottom
Louis Bergelson edited this page Nov 14, 2023 · 32 revisions

One-time setup tasks

In order to be able to release Picard, you first need to perform the following one-time setup tasks:

  • Set up your sonatype account

    • If you don't already have a sonatype account, create one by going to https://issues.sonatype.org/secure/Signup!default.jspa.

    • Check whether your sonatype account already has Deploy and UI Upload permissions for repo target com.github.broadinstitute by going to https://oss.sonatype.org/#stagingProfiles while logged in.

    • If you don't have Deploy and UI Upload permissions for com.github.broadinstitute, you need to request them. Ask someone who already has deployment permission to open a ticket on your behalf at https://issues.sonatype.org requesting permissions for you, and comment on the ticket yourself from your sonatype account once it's opened.

  • Set up your signing keys

    • Download GnuPG from http://www.gnupg.org/download/ if you don't already have it installed.

    • Depending on your set-up you may need to add the equivalent of the following two lines to your shell configuration file (e.g. .bashrc if using bash) so that you can input the key protecting passphrase whenever the gpg agent requests it:

      GPG_TTY=$(tty)
export GPG_TTY

      You may also need to restart the agent process (gpg-agent) if it was already up-and-running at the time you edit this file.

    • Run gpg --gen-key to generate a new keypair (NOTE: on some systems, the gpg command will be gpg1 or gpg2). Select the default values when asked for the kind and the size of the keys, and be sure to specify a passphrase for the keypair.

    • Run gpg --list-keys and gpg --list-secret-keys. The output should look something like this:

      $ gpg --list-keys
/Users/YOUR_USERNAME/.gnupg/pubring.gpg
---------------------------------
pub   4096R/4C1B2A36 2016-05-17
uid                  YOUR NAME <YOUR@EMAIL>
sub   4096R/26A37141 2016-05-17

    • Take note of the path to your secret keyring file (eg., /Users/YOUR_USERNAME/.gnupg/secring.gpg) and the public key id (4C1B2A36 in this case).

      NOTE: GnuGP 2.1 and later versions have changed the way they store secret keys (to individual files under ~/.gnupg/private-keys-v1.d) and so you will need to export these into the old secring.gpg format using the following command: gpg --export-secret-keys > ~/.gnupg/secring.gpg.

    • Publish your public key by running gpg --keyserver hkp://keyserver.ubuntu.com --send-keys YOUR_PUBLIC_KEY_ID, replacing YOUR_PUBLIC_KEY_ID with the key ID for your public key noted above (4C1B2A36 in the example). **NOTE: there'll also be a pre-existing GPGTools public key that comes with the software. Instead, you want to use/publish your new public key you created (here, and in properties below).

    • NOTE: If you are unable to do this command (--send-keys) and are getting a 'Network is unreachable', you may need to do this business:

      • Add the line standard-resolver to the file ~/.gnupg/dirmngr.conf (note that this file might not exist - yet)
      • Kill all dirmngr instances like this: killall dirmngr
      • Try the --send-keys command again.

  • Set up your credentials in gradle

    • Open ~/.gradle/gradle.properties in a text editor (creating it if necessary).

    • Add the following lines:

      signing.keyId=ID_FOR_YOUR_PUBLIC_KEY
signing.password=PASSPHRASE_FOR_YOUR_KEYPAIR
signing.secretKeyRingFile=PATH_TO_YOUR_SECRET_KEYRING_FILE
sonatypeUsername=YOUR_SONATYPE_USERNAME
sonatypePassword=YOUR_SONATYPE_PASSWORD

    • Save the file.

  • Ensure you have sudo access to picard on picard02.

    • ssh to picard02

    • sudo -u picard -i echo ok should print ok.

    • If you don't have access, contact one of the other picard developers for more information.

Releasing Picard

After you've performed the one-time setup steps above, here's what you need to do each time you want to release Picard:

  • Before releasing, make sure that tests are passing on master in travis.

  • In your git clone of Picard public, save any uncommitted work and run git checkout -f master. Running git diff HEAD should produce no output.

  • Run git pull --ff-only from your master branch to bring it up to date. Check the commit at the top of the git log output and make sure it matches the most recent commit at https://github.com/broadinstitute/picard/commits/master.

  • Run ./gradlew clean to clean your build directory.

  • Decide on a version number for the new Picard release (for example, 2.6.1). Typically we follow a very loose interpretation of semantic versioning, bumping the third number for bug fix releases (eg., 2.6.0 -> 2.6.1), the second number for releases with significant new features but no major breaking changes (eg., 2.6.1 -> 2.7.0), and the first number when there are major breaking changes (eg., 2.6.0 -> 3.0.0).

  • Tag the release by running git tag -a VERSION_NUMBER, where VERSION_NUMBER is the version you've chosen for the new release (eg., 2.6.1). Your text editor will pop up and ask you to write a commit message for the new tag. (This can just be the release number for now). After doing so, save and then exit. git describe should now print the new version.

  • Push the new tag by running git push origin VERSION_NUMBER, where VERSION_NUMBER is the tag you just created (for example, git push origin 2.6.1).

  • Run ./gradlew publish -Drelease=true. This builds the release and uploads it to sonatype's staging area.

  • Go to https://oss.sonatype.org/#stagingRepositories, logging in if necessary. If you don't see anything, click "refresh".

  • Find the release you just uploaded. It will probably be at the bottom with a name like comgithubbroadinstitute-1027, with your user ID listed as owner.

  • Check the box next to your release, then select "close". Press the refresh button repeatedly until it updates and its status says "closed".

  • Select your release again and click "release". Select the box to "automatically drop" in the pop-up confirmation dialog.

  • Wait ~30-180 minutes for the maven central release to happen. Once it happens, it will show up on https://search.maven.org/. Due to caching on maven's website it's often available by directly navigating to the coordinates sooner than it shows up in search.

  • Use this time to write release notes. Click on your new tag at https://github.com/broadinstitute/picard/tags, then click "Create release from tag", then add a list of the major changes since the last release. Don't click "publish release" just yet.

    • A good way to quickly auto-generate markdown-formatted release notes is with this command:

      git log --pretty=oneline PREVIOUS_VERSION..NEW_VERSION | grep -v "Merge pull request" | awk '{ $1=""; print "* " $0; }'

      Replacing PREVIOUS_VERSION and NEW_VERSION with the previous and current tags.

  • In your Picard git clone, run ./gradlew -Drelease=true clean shadowJar.

  • Back in the "edit" screen for your tag where you added release notes, click on "Attach binaries", and attach build/libs/picard.jar from your Picard clone.

  • Now with both release notes and a binary added to your tag on github, click "publish release" on the tag editing screen.

  • Go to https://github.com/broadinstitute/picard/releases and verify that everything looks ok, and also check that the release has shown up at https://search.maven.org/.

  • Once the release is available on the github releases page, ssh into picard02 and run:

    sudo -u picard -i /seq/pipeline/gppipeline-devtools/current/release_picard_public NEW_VERSION

    Replacing NEW_VERSION with the new version number.

  • Run ./build_push_docker.sh VERSION_NUMBER where VERSION_NUMBER is the tag being released. If fails pushing at very end run: docker push us.gcr.io/broad-gotc-prod/picard-cloud:NEW_VERSION

    This script builds and pushes a docker image to the GCR repo. The dockerhub image is automatically built using a dockerhub build rule.

  • Finally, run ./gradlew updateGhPages to publish updated documentation to https://broadinstitute.github.io/picard/.

  • Update the picard version in picard-private's build.gradle, and release picard-private

Clone this wiki locally