Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp release automation #1250

Merged
merged 26 commits into from
Aug 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
02de4a8
release: outline release logic using GoReleaser
hiddeco Jul 11, 2023
b5b3760
release: configure build of Docker images
hiddeco Jul 30, 2023
138d9fe
release: sign checksum file and images with cosign
hiddeco Jul 30, 2023
947979b
release: enable GitHub native note generation
hiddeco Jul 31, 2023
2186d1e
release: add release configuration
hiddeco Jul 31, 2023
9907309
release: publish container images to GHCR and Quay
hiddeco Jul 31, 2023
22fb5f0
release: download modules and check dirty state
hiddeco Jul 31, 2023
10c827d
build: outline new release workflow
hiddeco Jul 31, 2023
986a4c2
release: remove stale Make target utilities
hiddeco Aug 2, 2023
12c6e3f
release: small nits
hiddeco Aug 3, 2023
cf05f48
release: refactor backwards compatible artifacts
hiddeco Aug 3, 2023
0411c7d
release: extend timeout
hiddeco Aug 3, 2023
31823b0
release: further improve build flags
hiddeco Aug 4, 2023
30abe25
release: pin GoReleaser version
hiddeco Aug 4, 2023
0e585a1
release: artifact, SBOM and container provenance
hiddeco Aug 4, 2023
feb34a0
release: backwards compatible tag fmt, changelog
hiddeco Aug 8, 2023
fe45a84
release: further solve backwards compatible names
hiddeco Aug 8, 2023
feb7d43
build: remove obsolete Dockerfiles
hiddeco Aug 8, 2023
3deb00e
release: tweak signature and certificate filenames
hiddeco Aug 9, 2023
5276fcf
release: disable Go cache for workflow
hiddeco Aug 9, 2023
cd23b73
release: improve release notes
hiddeco Aug 9, 2023
bf884fa
release: remove DockerHub release util script
hiddeco Aug 10, 2023
13794d7
build: add `release-snapshot` Make target
hiddeco Aug 10, 2023
8f2333c
docs: briefly outline release procedure
hiddeco Aug 10, 2023
cbcd73d
release: update GoReleaser to 1.20.0
hiddeco Aug 10, 2023
79bbb22
release: further improvements based on feedback
hiddeco Aug 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .dockerignore

This file was deleted.

244 changes: 190 additions & 54 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,64 +2,200 @@ name: Release

on:
push:
tags:
- "v*"
tags: [ 'v*' ]

permissions:
contents: read

jobs:
tagged-release:
name: "Tagged Release"
release:
runs-on: ubuntu-latest

permissions:
contents: write # For creating the GitHub release.
id-token: write # For creating OIDC tokens for signing.
packages: write # For pushing and signing container images.

outputs:
artifact-subjects: "${{ steps.artifact-hashes.outputs.subjects }}"
package-subjects: "${{ steps.package-hashes.outputs.subjects }}"
sbom-subjects: "${{ steps.sbom-hashes.outputs.subjects }}"
container-subjects: "${{ steps.container-metadata.outputs.subjects }}"

steps:
- name: Install dependencies
run: sudo apt-get update && sudo apt-get install git ruby rpm -y
- name: Install fpm
run: gem install fpm || sudo gem install fpm
- name: Set up Go 1.20
uses: actions/setup-go@v3
- name: Checkout
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
with:
fetch-depth: 0

- name: Setup Go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1
with:
go-version: 1.20.x
cache: false

- name: Setup Syft
uses: anchore/sbom-action/download-syft@78fc58e266e87a38d4194b2137a3d4e9bcaf7ca1 # v0.14.3

- name: Setup Cosign
uses: sigstore/cosign-installer@6e04d228eb30da1757ee4e1dd75a0ec73a653e06 # v3.1.1

- name: Setup QEMU
uses: docker/setup-qemu-action@2b82ce82d56a2a04d2637cd93a637ae1b359c0a7 # v2.2.0

- name: Setup Docker Buildx
uses: docker/setup-buildx-action@4c0219f9ac95b02789c1075625400b2acbff50b1 # v2.9.1

- name: Login to GitHub Container Registry
uses: docker/login-action@465a07811f14bebb1938fbed4728c6a1ff8901fc # v2.2.0
with:
go-version: '1.20'
id: go
- name: Check out code into the Go module directory
uses: actions/checkout@v3
- name: Go vendor
run: go mod vendor
- name: Make release directory
run: mkdir dist
- name: Build deb and rpm
run: make deb-pkg rpm-pkg
- name: Move deb and rpm into release directory
run: mv *.deb *.rpm dist/
- name: Set RELEASE_VERSION
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
- name: Set RELEASE_NUMBER
run: echo "RELEASE_NUMBER=$(echo $RELEASE_VERSION | cut -c2-)" >> $GITHUB_ENV
- name: Build linux amd64 binary
run: GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -mod vendor -o dist/sops-${{ env.RELEASE_VERSION }}.linux.amd64 github.com/getsops/sops/v3/cmd/sops && cp dist/sops-${{ env.RELEASE_VERSION }}.linux.amd64 dist/sops-${{ env.RELEASE_VERSION }}.linux
- name: Build linux arm64 binary
run: GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -mod vendor -o dist/sops-${{ env.RELEASE_VERSION }}.linux.arm64 github.com/getsops/sops/v3/cmd/sops
- name: Build darwin amd64 binary
run: GOOS=darwin GOARCH=amd64 CGO_ENABLED=0 go build -mod vendor -o dist/sops-${{ env.RELEASE_VERSION }}.darwin.amd64 github.com/getsops/sops/v3/cmd/sops
- name: Copy darwin amd64 to have a no-architecture labeled version
run: cp dist/sops-${{ env.RELEASE_VERSION }}.darwin.amd64 dist/sops-${{ env.RELEASE_VERSION }}.darwin
- name: Build darwin arm64 binary
run: GOOS=darwin GOARCH=arm64 CGO_ENABLED=0 go build -mod vendor -o dist/sops-${{ env.RELEASE_VERSION }}.darwin.arm64 github.com/getsops/sops/v3/cmd/sops
- name: Build windows binary
run: GOOS=windows CGO_ENABLED=0 go build -mod vendor -o dist/sops-${{ env.RELEASE_VERSION }}.exe github.com/getsops/sops/v3/cmd/sops
- name: Create release
uses: "mozilla/action-automatic-releases@latest"
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Login to Quay.io
uses: docker/login-action@465a07811f14bebb1938fbed4728c6a1ff8901fc # v2.2.0
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
prerelease: true
files: |
dist/sops-${{ env.RELEASE_VERSION }}.exe
dist/sops-${{ env.RELEASE_VERSION }}.darwin.amd64
dist/sops-${{ env.RELEASE_VERSION }}.darwin.arm64
dist/sops-${{ env.RELEASE_VERSION }}.darwin
dist/sops-${{ env.RELEASE_VERSION }}.linux.amd64
dist/sops-${{ env.RELEASE_VERSION }}.linux.arm64
dist/sops-${{ env.RELEASE_VERSION }}.linux
dist/sops_${{ env.RELEASE_NUMBER }}_amd64.deb
dist/sops_${{ env.RELEASE_NUMBER }}_arm64.deb
dist/sops-${{ env.RELEASE_NUMBER }}-1.x86_64.rpm
dist/sops-${{ env.RELEASE_NUMBER }}-1.aarch64.rpm
registry: quay.io
username: ${{ secrets.QUAY_BOT_USERNAME }}
password: ${{ secrets.QUAY_BOT_TOKEN }}

- name: Run GoReleaser
id: goreleaser
uses: goreleaser/goreleaser-action@336e29918d653399e599bfca99fadc1d7ffbc9f7 # v4.3.0
with:
version: 1.20.x
args: release --clean --timeout 1h
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Extract artifact subjects
id: artifact-hashes
env:
ARTIFACTS: "${{ steps.goreleaser.outputs.artifacts }}"
run: |
set -euo pipefail
sum_file=$(echo -E $ARTIFACTS | jq -r '.[] | {name, "digest": (.extra.Digest // .extra.Checksum)} | select(.digest) | {digest} + {name} | join(" ") | sub("^(.*?):";"")')
echo "subjects=$(echo "$sum_file" | base64 -w0)" >> "$GITHUB_OUTPUT"

- name: Extract package subjects
id: package-hashes
env:
ARTIFACTS: "${{ steps.goreleaser.outputs.artifacts }}"
run: |
set -euo pipefail

sum_file="$(mktemp)"

mapfile -t file_paths < <(echo -E "$ARTIFACTS" | jq -r '.[] | select(.type=="Linux Package") | .path')
for f in "${file_paths[@]}"; do
file_name=$(basename "$f")
file_sum=$(sha256sum "$f" | awk '{print $1}')
echo "$file_sum $file_name" >> "$sum_file"
done

echo "subjects=$(base64 -w0 < "$sum_file")" >> "$GITHUB_OUTPUT"

- name: Extract SBOM subjects
id: sbom-hashes
env:
ARTIFACTS: "${{ steps.goreleaser.outputs.artifacts }}"
run: |
set -euo pipefail

sum_file="$(mktemp)"

mapfile -t file_paths < <(echo -E "$ARTIFACTS" | jq -r '.[] | select(.type=="SBOM") | .path')
for f in "${file_paths[@]}"; do
file_name=$(basename "$f")
file_sum=$(sha256sum "$f" | awk '{print $1}')
echo "$file_sum $file_name" >> "$sum_file"
done

echo "subjects=$(base64 -w0 < "$sum_file")" >> "$GITHUB_OUTPUT"

- name: Extract container image subjects
id: container-metadata
env:
ARTIFACTS: "${{ steps.goreleaser.outputs.artifacts }}"
run: |
image_list=$(echo -e "$ARTIFACTS" | jq -r '.[] | select(.type=="Docker Manifest") | {"image": (.name | sub("^.*?/"; "") | sub(":(.*)"; "")), "digest": .extra.Digest}')
echo "subjects=$(echo $image_list | jq -c -s 'unique_by(.digest) | {"include": .}')" >> "$GITHUB_OUTPUT"

combine-subjects:
runs-on: ubuntu-latest

needs: [ release ]

outputs:
all-subjects: "${{ steps.combine-subjects.outputs.subjects }}"

steps:
- name: Combine subjects
id: combine-subjects
env:
ARTIFACT_SUBJECTS: "${{ needs.release.outputs.artifact-subjects }}"
PACKAGE_SUBJECTS: "${{ needs.release.outputs.package-subjects }}"
SBOM_SUBJECTS: "${{ needs.release.outputs.sbom-subjects }}"
run: |
set -euo pipefail

artifact_subjects=$(echo "$ARTIFACT_SUBJECTS" | base64 -d)
package_subjects=$(echo "$PACKAGE_SUBJECTS" | base64 -d)
sbom_subjects=$(echo "$SBOM_SUBJECTS" | base64 -d)

all_subjects=$(echo -e "${artifact_subjects}\n${package_subjects}\n${sbom_subjects}\n" | sed '/^$/d')

echo "subjects=$(echo "$all_subjects" | base64 -w0)" >> "$GITHUB_OUTPUT"

assets-provenance:
needs: [ combine-subjects ]

permissions:
actions: read # For detecting the GitHub Actions environment.
id-token: write # For creating OIDC tokens for signing.
contents: write # For adding assets to a release.

uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v1.8.0
with:
base64-subjects: "${{ needs.combine-subjects.outputs.all-subjects }}"
upload-assets: true
provenance-name: "provenance.intoto.jsonl"

ghcr-container-provenance:
needs: [ release ]

permissions:
actions: read # For detecting the Github Actions environment.
id-token: write # For creating OIDC tokens for signing.
packages: write # For uploading attestations.

strategy:
matrix: ${{ fromJSON(needs.release.outputs.container-subjects) }}

uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v1.8.0
with:
image: ghcr.io/${{ matrix.image }}
digest: ${{ matrix.digest }}
registry-username: ${{ github.actor }}
secrets:
registry-password: ${{ secrets.GITHUB_TOKEN }}

quay-container-provenance:
needs: [ release ]

permissions:
actions: read # For detecting the Github Actions environment.
id-token: write # For creating OIDC tokens for signing.
packages: write # For uploading attestations.

strategy:
matrix: ${{ fromJSON(needs.release.outputs.container-subjects) }}

uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v1.8.0
with:
image: quay.io/${{ matrix.image }}
digest: ${{ matrix.digest }}
secrets:
registry-username: ${{ secrets.QUAY_BOT_USERNAME }}
registry-password: ${{ secrets.QUAY_BOT_TOKEN }}
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
target
bin/
dist/
Cargo.lock
vendor/
coverage.txt
Expand Down
Loading