Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to support PEP 730-format support packages. #55

Merged
merged 16 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: CI
on:
pull_request:
push:
branches:
- main
workflow_call:

# Cancel active CI runs for a PR before starting another run
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

defaults:
run:
shell: bash

env:
FORCE_COLOR: "1"

jobs:
pre-commit:
name: Pre-commit checks
uses: beeware/.github/.github/workflows/pre-commit-run.yml@main
with:
pre-commit-source: "pre-commit"

test:
name: Test builds
needs: pre-commit
runs-on: macOS-latest
strategy:
fail-fast: false
matrix:
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13-dev" ]

steps:
- name: Checkout
uses: actions/checkout@v4.1.7
with:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v5.2.0
with:
python-version: ${{ matrix.python-version }}

# Initial call to the setup script sets up the environment
- name: Set up Forge
run: |
source ./setup-iOS.sh $(cut -d- -f1 <<< ${{ matrix.python-version }})

# Build an example of a native package
# Calling setup script activates existing environment
- name: Build libjpeg
run: |
source ./setup-iOS.sh $(cut -d- -f1 <<< ${{ matrix.python-version }})
forge iOS libjpeg

# Build an example of a simple Python package
# Calling setup script activates existing environment
- name: Build lru-dict
run: |
source ./setup-iOS.sh $(cut -d- -f1 <<< ${{ matrix.python-version }})
forge iOS lru-dict
12 changes: 12 additions & 0 deletions .github/workflows/pre-commit-update.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: Update pre-commit

on:
schedule:
- cron: "0 20 * * SUN" # Sunday @ 2000 UTC
workflow_dispatch:

jobs:
pre-commit-update:
name: Update pre-commit
uses: beeware/.github/.github/workflows/pre-commit-update.yml@main
secrets: inherit
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,12 @@
__pycache__
*.log
*.DS_Store
deps/
downloads/
dist/
venv*/
build/
local/
published/
tools/
support/
16 changes: 8 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.6.0
hooks:
- id: check-toml
# Recipes are templates, not pure YAML
Expand All @@ -10,31 +10,31 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort
additional_dependencies: [toml]
- repo: https://github.com/asottile/pyupgrade
rev: v3.11.0
rev: v3.17.0
hooks:
- id: pyupgrade
args: [--py38-plus]
args: [--py39-plus]
- repo: https://github.com/PyCQA/docformatter
rev: v1.7.5
hooks:
- id: docformatter
args: [--in-place, --black]
- repo: https://github.com/psf/black
rev: 23.9.1
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 24.8.0
hooks:
- id: black
language_version: python3
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
rev: 7.1.1
hooks:
- id: flake8
- repo: https://github.com/codespell-project/codespell
rev: v2.2.5
rev: v2.3.0
hooks:
- id: codespell
additional_dependencies:
Expand Down
105 changes: 43 additions & 62 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,92 +1,80 @@
Mobile Forge
============

This is a forge-like environment that can be used to build wheels for mobile platforms.
It is currently only tested for iOS, but in theory, it should also be usable for
Android. Contributions to verify Android support, and to add more package recipes, are
definitely encouraged.
This is a forge-like environment that can be used to build wheels for mobile
platforms. It is currently only tested for iOS, but in theory, it should also be
usable for Android. Contributions to verify Android support, tvOS and watchOS
support, and to add more package recipes, are enthusiastically encouraged.

Usage
-----

Before using Mobile Forge, you'll need to compile Python for your build platform (e.g.,
your laptop), and for host platform (e.g., for iOS). It may be helpful to use a project
like `Python-Apple-support <https://github.com/beeware/Python-Apple-support>`__ to
manage this compilation process.
This repo contains an activation script that will configure your environment so
it's ready to use. To set up a build environment:

Using Python-Apple-support
~~~~~~~~~~~~~~~~~~~~~~~~~~

If you *do* use Python-Apple-support, this repo contains an activation script that will
configure your environment so it's ready to use.

1. Set an environment variable declaring the location of your Python-Apple-support
checkout::

$ export PYTHON_APPLE_SUPPORT=/path/to/Python-Apple-support

2. Clone this repository, and run the activate script for the Python version you want to
use::
1. Clone this repository::

$ git clone https://github.com/beeware/mobile-forge.git
$ cd mobile-forge

2. Run the script for the Python version you want to use, providing the support
revision::

$ source ./setup-iOS.sh 3.11

This will create a Python virtual environment, install mobile forge, and provide
some hints at forge commands you can run.
Running this script will create a Python virtual environment, install Mobile
Forge and some other required tools, and provide some hints at forge commands
you can run.

If a virtual environment already exists, it will be activated, and the same hints
displayed.

The hard way
~~~~~~~~~~~~

If you're *not* using Python-Apple-support, the setup process requires more manual steps::

1. Create and activate a virtual environment using the Python build platform, using the
build platform Python compiled in step 1.
``lru-dict`` is a good first package to try compiling::

2. Clone this repository, and install it into your freshly created virtual environment::
(venv3.11) $ forge iOS lru-dict

(venv3.11) $ git clone https://github.com/beeware/mobile-forge.git
(venv3.11) $ cd mobile-forge
(venv3.11) $ pip install -e .
Or, to build a wheel for a single architecture::

3. Ensure your ``PATH`` contains any tools that were necessary to compile the host CPython,
and does *not* contain any macOS development libraries. Mobile-forge will clean the ``PATH``
to remove known problematic paths (e.g., paths added by Homebrew, rbenv, npm, etc).
(venv3.11) $ forge iphonesimulator:12.0:arm64 lru-dict

4. Set environment variables that define the location of the Python executable for each
of the **host** platforms you intend to target. For iOS, this means defining
3 environment variables::
Once this command completes, there should be a wheel for each platform in the ``dist``
folder. A log for each successful build will be in the ``logs`` folder; a log for each
unsuccessful build (if there are any) will be in the ``errors`` folder.

(venv3.11) $ export MOBILE_FORGE_IPHONEOS_ARM64=...
(venv3.11) $ export MOBILE_FORGE_IPHONESIMULATOR_ARM64=...
(venv3.11) $ export MOBILE_FORGE_IPHONESIMULATOR_X86_64=...
Local support package builds
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

5. Build a package. The ``packages`` folder contains recipes for packages. ``lru-dict``
is a good first package to try::
By default, the Mobile Forge setup script will download a support revision and
use the binaries in the downloaded package. However, you can also use a local
build of the support package.

(venv3.11) $ forge iOS lru-dict
After cloning and building `Python-Apple-support
<https://github.com/beeware/Python-Apple-support>`__, set the
``PYTHON_APPLE_SUPPORT`` environment variable to the root of the
Python-Apple-support checkout. Then run the ``setup-iOS.sh`` script to configure
your environment.

Or, to build a wheel for a single architecture::
Specific support package builds
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(venv3.11) $ forge iphonesimulator:12.0:arm64 lru-dict
The Mobile Forge setup script will download a support package for any supported
Python version. The version that is downloaded is hard-coded in the setup
script. To use a specific revision rather than the default, add the revision
number as an additional argument to the setup script. For example, to use
revision 4 of the 3.11 support package, run::

Once this command completes, there should be a wheel for each platform in the ``dist``
folder. A log for each successful build will be in the ``logs`` folder; a log for each
unsuccessful build (if there are any) will be in the ``errors`` folder.
$ source ./setup-iOS.sh 3.11 4

The special snowflakes
~~~~~~~~~~~~~~~~~~~~~~
----------------------

Mobile Forge is trying to support multiple packages, building on multiple Python
versions, for multiple architectures; and some of those Python versions were released
before the release of ARM64 macOS hardware. As a result, some versions of some packages
have some quirks that must be taken into account.

Pandas
^^^^^^
~~~~~~

Pandas uses a meta-package named ``oldest-supported-numpy`` to ensure ABI compatibility
during compilation. However, this can install a different version of numpy, depending on
Expand All @@ -97,17 +85,10 @@ as version 2999.1.1, which ensures that consistent versions are available for bu
purposes; however, this wheel *should not* be published.

Cryptography
^^^^^^^^^^^^
~~~~~~~~~~~~

Cryptography currently builds a *very* old version (3.4.8). This is the last version
that could be built without a Rust compiler. The recipe works as is on all Python
versions *except* Python 3.8 on ARM64, because there was no ARM64-compatible wheel
published for cffi 1.15.1. However, if you run::

$ pip wheel -w dist --no-deps cffi==1.15.1

you can build a universal Python3.8 CFFI wheel for CFFI 1.15.1, which can be used to
satisfy this build-time requirement.
that could be built without a Rust compiler.

What now?
---------
Expand Down
Loading