Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: improve license update action #593

Merged
merged 10 commits into from
Apr 9, 2024
96 changes: 77 additions & 19 deletions .github/workflows/update_licenses.yaml
Original file line number Diff line number Diff line change
@@ -1,22 +1,46 @@
# Workflow to update licenses for x86 Linux and MacOS
# Support ARM MacOS
# FIXME: https://github.com/zama-ai/concrete-ml-internal/issues/3925
name: Update licenses
on:
workflow_dispatch:
inputs:
linux:
description: "Update licenses for Ubuntu"
type: boolean
required: false
default: true
mac_silicon:
description: "Update licenses for macOS (silicon)"
type: boolean
required: false
default: true
mac_intel:
description: "Update licenses for macOS (intel)"
type: boolean
required: false
default: true

concurrency:
group: "${{ github.ref }}-${{ github.event_name }}-${{ github.workflow }}"
cancel-in-progress: false
cancel-in-progress: true

jobs:
# Update licenses for all supported OS
update_licenses:
strategy:
matrix:
# No arm-macos machines on github runners
# we would need to use one of our own runners
os: [ubuntu-20.04, macos-latest-xl]
runs-on: ${{ matrix.os }}
config:
- os_name: linux
os: ubuntu-latest
if: ${{ inputs.linux }}
- os_name: mac_silicon
os: macos-latest-xlarge
if: ${{ inputs.mac_silicon }}
- os_name: mac_intel
os: macos-latest-large
if: ${{ inputs.mac_intel }}
fail-fast: false

runs-on: ${{ matrix.config.os }}
defaults:
run:
shell: bash
Expand All @@ -32,16 +56,18 @@ jobs:
echo "::add-mask::${{ secrets.INTERNAL_PYPI_URL_FOR_MASK }}"
echo "::add-mask::${{ secrets.INTERNAL_REPO_URL_FOR_MASK }}"

# Checkout repository
- name: Checkout Code
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
with:
token: ${{ secrets.BOT_TOKEN }}

- name: Set up Python 3.8
# Set up python 3.10 because Github does not seem to provide support for python 3.8 on arm64
# List of available versions and architectures :
# https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
- name: Set up Python 3.10
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d
with:
python-version: '3.8'
python-version: '3.10'

- name: Install dependencies
run: |
Expand All @@ -52,11 +78,43 @@ jobs:
- name: Update licenses
run: |
make licenses

# Pull the latest changes if there are some
- name: Pull latest changes

# Upload the updated license files as artifacts, if they exist
# It is possible that no files have been generated if licenses have already been done for some
# configuration(s)
- uses: actions/upload-artifact@v4
with:
if-no-files-found: ignore
name: licenses_${{ matrix.config.os_name }}
path: |
deps_licenses/licenses_${{ matrix.config.os_name }}_user.txt
deps_licenses/licenses_${{ matrix.config.os_name }}_user.txt.md5

# Push the updates license files, as a PR or directly to the branch
push_licenses:
runs-on: ubuntu-latest
needs: [update_licenses]
if: always()
steps:
# Mask internal URLs if logged
- name: Add masks
id: masks
run: |
git pull -X theirs
echo "::add-mask::${{ secrets.INTERNAL_PYPI_URL_FOR_MASK }}"
echo "::add-mask::${{ secrets.INTERNAL_REPO_URL_FOR_MASK }}"

- name: Checkout Code
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
with:
token: ${{ secrets.BOT_TOKEN }}

# Retrieve all updated license files
# Enable merge multiple to download all files in the 'deps_licenses' directory
- name: Download artifacts
uses: actions/download-artifact@v4
with:
path: deps_licenses/
merge-multiple: true

# If the target branch is main or a release branch, a pull request is opened for everyone to
# review
Expand All @@ -65,16 +123,16 @@ jobs:
uses: peter-evans/create-pull-request@70a41aba780001da0a30141984ae2a0c95d8704e
with:
token: ${{ secrets.BOT_TOKEN }}
commit-message: "chore: update licenses ${{ matrix.os }}"
branch: "chore/update_licenses_${{ matrix.os }}"
commit-message: "chore: update licenses"
branch: "chore/update_licenses"
base: "${{ github.ref_name }}"
title: "Update licenses for ${{ matrix.os }} on ${{ github.ref_name }}"
body: "Update licenses for ${{ matrix.os }} on ${{ github.ref_name }}"
title: "Update licenses in ${{ github.ref_name }}"
body: "Update licenses in ${{ github.ref_name }}"

# If the target branch is another branch, the current branch is automatically merged into it
- name: Push changes into the current branch
if: ${{ github.ref_name != 'main' && !(startsWith(github.ref_name , 'release/')) }}
uses: stefanzweifel/git-auto-commit-action@v5
with:
commit_message: "chore: update licenses for ${{ matrix.os }}"
commit_message: "chore: update licenses"
add_options: '-u'
4 changes: 3 additions & 1 deletion script/make_utils/fix_omp_issues_for_intel_mac.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ set -ex

UNAME=$(uname)
MACHINE=$(uname -m)
PYTHON_VERSION=$(python --version | cut -d' ' -f2 | cut -d'.' -f1,2)

if [ "$UNAME" == "Darwin" ] && [ "$MACHINE" != "arm64" ]
# The error does not seem to happen on python 3.10 (on MacOS 13.6.6)
if [ "$UNAME" == "Darwin" ] && [ "$MACHINE" != "arm64" ] && [ "$PYTHON_VERSION" != "3.10" ]
then

# We need to source the venv here, since it's not done in the CI
Expand Down
Loading