3.22.0rc0

CCSI-FOQUS: cutting release 3.22.0rc0

Create and/or update 3.22_rel release branch

• 1: Clone repository in a fresh directory

  pushd "$(mktemp -d)"
  git clone git@github.com:CCSI-Toolset/FOQUS.git && cd FOQUS

• 2: Switch to release branch if it exists, else create it

  git checkout "3.22_rel" || git checkout -b "3.22_rel"

• 3: Review differences with the default branch

  • Go to compare URL https://github.com/CCSI-Toolset/FOQUS/compare/3.22_rel...master

• 4: Cherry-pick commits onto the release branch (N/A)

• 5: Update version and dependencies in ./setup.py (WaterTAP, DISPATCHES) (N/A)

• 6: Update version and dependencies (FOQUS)

  • Open ./setup.py
  • Search for default_version = and set default_version = "3.22.0rc0"
  • Save and exit
  • Open ./docs/source/conf.py
  • Search for version = and set version = "3.22.0rc0"
  • Search for release = and set release = "3.22.0rc0"
  • Save and exit
  • git add setup.py ./docs/source/conf.py

• 7: Update version (IDAES) (N/A)

• 8: Update version (IDAES examples) (N/A)

• 9: Update version (IDAES examples-pse) (N/A)

• 10: Update version and dependencies in ./setup.py (PARETO) (N/A)

• 11: Check that the local modifications to the version are as they should

  git status      # there shouldn't be any other unstaged files
  git status -vv  # the changes with the version should be there

• 12: Commit the changes to the 3.22_rel branch

  # check that we're on the correct release branch
  test "$(git branch --show-current)" = "3.22_rel" && git commit -m "3.22.0rc0"

• 13: Review the changes before pushing

  git log --oneline -n 5  # it should show only one commit which is not pushed
  git push --set-upstream git@github.com:CCSI-Toolset/FOQUS.git "3.22_rel" --dry-run

• 14: Push the changes

  git push --set-upstream git@github.com:CCSI-Toolset/FOQUS.git "3.22_rel"

Updating default branch with next dev version

• 1: Clone repository in a fresh directory

  pushd "$(mktemp -d)"
  git clone git@github.com:CCSI-Toolset/FOQUS.git && cd FOQUS

• 2: Update version and dependencies in ./setup.py (FOQUS)

  • Open ./setup.py
  • Search for default_version = and set default_version = "3.23.dev0"
  • Save and exit

• 3: Update version (IDAES) (N/A)

• 4: Update version (IDAES examples) (N/A)

• 5: Update version (IDAES examples-pse) (N/A)

• 6: Update version in ./docs/source/conf.py

  • Open ./docs/source/conf.py
  • Search for version = and set version = "3.23.dev0"
  • Search for release = and set release = "3.23.dev0"
  • Save and exit

• 7: Update version and dependencies in ./setup.py (PARETO) (N/A)

• 8: Update version and dependencies in ./setup.py and docs/conf.py (WaterTAP, DISPATCHES) (N/A)

• 9: Check that the local modifications to the version are as they should

  git status      # there shouldn't be any other unstaged files
  git status -vv  # the changes with the version should be there

• 10: Commit the changes to the default branch

  # check that we're on the default branch
  test "$(git branch --show-current)" = "master" && git commit -m "3.23.dev0"

• 11: Review the changes before pushing

  git log --oneline -n 5  # it should show only one commit which is not pushed
  git push --dry-run

• 12: Push the changes

  git push

Creating the GitHub release

• 1: Generate release notes and create GitHub draft release Copy and paste the following release notes into a file named release-notes-3.22.0rc0.md:

  # 3.22 Release Candidate #0
  
  Start with our [online documentation](https://foqus.readthedocs.org/en/3.22.0rc0) to get started with install instructions, examples, etc.
  
  Note: Download the below `ccsi-foqus-3.22.0rc0-examples.zip` file for all the CCSI-FOQUS examples, tutorials and test files.
  
  ## 3.22.0rc0 Release Highlights:
  

  Or, run the following shell command to create the file in the local directory:

  cat <<'EOF' > release-notes-3.22.0rc0.md
  # 3.22 Release Candidate #0
  
  Start with our [online documentation](https://foqus.readthedocs.org/en/3.22.0rc0) to get started with install instructions, examples, etc.
  
  Note: Download the below `ccsi-foqus-3.22.0rc0-examples.zip` file for all the CCSI-FOQUS examples, tutorials and test files.
  
  ## 3.22.0rc0 Release Highlights:
  
  
  EOF

  Run this command to create a draft release using the gh CLI tool

  gh release create "3.22.0rc0" --repo "CCSI-Toolset/FOQUS" --target "3.22_rel" --title "3.22 Release Candidate #0" --notes-file "release-notes-3.22.0rc0.md" --prerelease --draft

• 2: Create a ZIP file for the examples and attach it to the release as an asset

  _asset_file_name="ccsi-foqus-3.22.0rc0-examples.zip"
  zip -r "$_asset_file_name" examples/
  gh release upload "3.22.0rc0" "$_asset_file_name" --repo "CCSI-Toolset/FOQUS"

• 3: Update the compatibility file on main (N/A)

• 4: Check that the GitHub release was created successfully

  • Go to https://github.com/CCSI-Toolset/FOQUS/releases/tag/3.22.0rc0
  • If "Draft", click on https://github.com/CCSI-Toolset/FOQUS/releases/edit/3.22.0rc0 to remove the "Draft" marker

• 5: Check that the release tag has been created in the repo

  curl -sL https://github.com/CCSI-Toolset/FOQUS/archive/3.22.0rc0.zip | sha256sum

  NOTE

  • The release needs to be undrafted for this to work
  • Use wget if curl -sL doesn't work

Deleting the release (if needed)

• 1: Delete the release on GitHub

  # add the --yes flag to skip confirmation prompt
  gh release delete --repo CCSI-Toolset/FOQUS "3.22.0rc0"

• 2: Delete the tag on the remote

  pushd "$(mktemp -d)"
  git clone --depth 1 --branch "3.22_rel" https://github.com/CCSI-Toolset/FOQUS && cd FOQUS
  git push --delete git@github.com:CCSI-Toolset/FOQUS.git "refs/tags/3.22.0rc0"

• 3: Delete the tag locally

  git tag --delete "3.22.0rc0"

Checking the docs (ReadTheDocs)

• 1: Check the ReadTheDocs build

  • Go to https://www.readthedocs.org/projects/foqus/builds and check that the build for 3.22.0rc0 has been run successfully
  • If not, edit the version at https://www.readthedocs.org/dashboard/foqus/version/3.22.0rc0/edit so that it starts building
  • If the previous step didn't work, go to https://www.readthedocs.org/projects/foqus/versions, search for the 3.22.0rc0 version, and click on "Edit"

• 2: Check that the 3.22.0rc0 tag is available on ReadTheDocs

  • Manually, at https://foqus.readthedocs.org/en/3.22.0rc0
  • curl -sL "https://foqus.readthedocs.org/en/3.22.0rc0" | grep "/3.22.0rc0/"
  • curl -sL "https://foqus.readthedocs.org/en/3.22.0rc0" | grep "Versions" --after 10 | grep "/3.22.0rc0/"

• 3: Check that the ReadTheDocs revision (commit) on latest matches the release tag

  curl -sL "https://foqus.readthedocs.org/en/3.22.0rc0" | grep "Revision"
  curl -sL "https://foqus.readthedocs.org/en/latest" | grep "Revision"
  curl -sL "https://foqus.readthedocs.org" | grep "Revision"

• 4: Check that the ReadTheDocs revision (commit) on stable matches the release tag (N/A)

Creating the ccsi-foqus Python package distribution

• 1: In a clean directory, clone the repo and switch to the release branch

  pushd "$(mktemp -d)"
  git clone --depth 1 --branch "3.22_rel" https://github.com/CCSI-Toolset/FOQUS && cd FOQUS
  # check that the branch is at the 3.22.0rc0 tag
  git tag --list | grep "3.22.0rc0"

• 2: Create a new Conda environment for building the package and activate it

  # bash/zsh
  _build_env_name="build-ccsi-foqus-3.22.0rc0"
  conda create --yes python=3.8 pip setuptools wheel twine --name "$_build_env_name" && conda activate "$_build_env_name"
  # test that the pip executable is the one from the environment
  which pip | grep "$_build_env_name"

• 3: Install the package (NO EDITABLE MODE)

  pip install --no-cache-dir .

• 4: Check that the installed package has the proper tags

  pip list | grep "3.22.0rc0"
  pip show ccsi-foqus | grep "3.22.0rc0"

• 5: Run "smoke tests" for the installed package (once uploaded to PyPI, IT CANNOT BE DELETED!)

  foqus -w "$(mktemp -d)"

• 6: Remove large directories to avoid exceeding size limits

  rm -r ./examples/

• 7: Build the sdist and bdist_wheel distributions

  python setup.py sdist bdist_wheel

• 8: Check that the directory size is below PyPI's maximum

  ls -lh dist/*
  _maxsize=100000  # 100 MB
  _dirsize=$(du -s dist/ | cut -f 1) ; echo $_dirsize
  [ "$_dirsize" -lt "$_maxsize" ]

• 9: Upload the distributions to the test PyPI

  # bash/zsh
  # assumes that the ~/.pypirc config file exists and defines the "testpypi" repository
  grep "\[testpypi\]" ~/.pypirc && twine upload --repository testpypi dist/*

  NOTE It might take a while for the uploaded package to become available for pip install

• 10: Test package from test PyPI in a dedicated temp Conda environment

  _test_env_name="test-ccsi-foqus-3.22.0rc0"
  conda create --yes python=3.8 --name "$_test_env_name" && conda activate "$_test_env_name"
  which pip | grep "$_test_env_name"
  pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple "ccsi-foqus==3.22.0rc0"
  foqus -w "$(mktemp -d)"
  conda deactivate && conda env remove --name "$_test_env_name"

• 11: Upload the distributions to the real PyPI

  conda activate "$_build_env_name"
  # assumes that the ~/.pypirc config file exists and defines the "realpypi" repository
  grep "\[realpypi\]" ~/.pypirc && twine upload --repository realpypi dist/*

  NOTE It might take a while for the uploaded package to become available for pip install

• 12: Install package from real PyPI and generate requirements/environment files

  # bash/zsh
  _rel_prefix="ccsi-foqus-3.22.0rc0"
  _env_name="${_rel_prefix}-env"
  _conda_file_name="environment-${_rel_prefix}.yml"
  _pip_file_name="requirements-${_rel_prefix}.txt"
  conda create --yes python=3.8 --name "$_env_name" && conda activate "$_env_name"
  which pip | grep "$_env_name"
  pip install "ccsi-foqus==3.22.0rc0"
  pushd "$(mktemp -d)"
  conda env export -n "$_env_name" -f "$_conda_file_name"
  pip freeze > "$_pip_file_name"
  gh release upload "3.22.0rc0" "$_conda_file_name" "$_pip_file_name" --repo "CCSI-Toolset/FOQUS"
  conda deactivate && popd

Build and deploy examples using containers (N/A)

Build and deploy examples (N/A)

Collect environment info for the release

• 1: Install from PyPI

  # bash/zsh
  _rel_prefix="ccsi-foqus-3.22.0rc0"
  _env_name="${_rel_prefix}-env"
  _conda_file_name="environment-${_rel_prefix}.yml"
  _pip_file_name="requirements-${_rel_prefix}.txt"
  conda create --yes python=3.8 --name "$_env_name" && conda activate "$_env_name"
  which pip | grep "$_env_name"
  pip install "ccsi-foqus==3.22.0rc0"

• 2: Save environment info

  pushd "$(mktemp -d)"
  conda env export --no-builds -n "$_env_name" -f "$_conda_file_name"
  pip freeze > "$_pip_file_name"
  cat "$_conda_file_name" "$_pip_file_name"

• 3: Upload as GitHub release assets

  gh release upload "3.22.0rc0" "$_conda_file_name" "$_pip_file_name" --repo "CCSI-Toolset/FOQUS"
  conda deactivate && conda env remove -n "$_env_name" && popd

• 4: Check that the assets are available

  • https://github.com/CCSI-Toolset/FOQUS/releases/tag/3.22.0rc0

Announce the release

• 1: Internally Summary text (in markdown):

  Release 3.22.0rc0 is up!
  
  - GitHub: https://github.com/CCSI-Toolset/FOQUS/releases/tag/3.22.0rc0
  - PyPI: https://pypi.org/project/ccsi-foqus/3.22.0rc0/
  - ReadTheDocs: https://foqus.readthedocs.org/en/3.22.0rc0

  ---

  Release 3.22.0rc0 is up!

  • GitHub: https://github.com/CCSI-Toolset/FOQUS/releases/tag/3.22.0rc0
  • PyPI: https://pypi.org/project/ccsi-foqus/3.22.0rc0/
  • ReadTheDocs: https://foqus.readthedocs.org/en/3.22.0rc0

  ---

• 2: Announce the release

  • Via an email to the leadership, lab and users list (TODO: add template)