Build speed and runners

[withinboredom]

I took my fork of FrankenPHP and set it up for self-hosted runners. I have a decently big Kubernetes cluster for self-hosted builds on some internal stuff, so it is idle quite a bit. In saying that, I was curious if using the cluster would speed up build times or not.

In essence, it does speed things up somewhat but also slows things down. These servers have somewhat limited bandwidth, and most of the build is spent downloading and building tooling to actually build FrankenPHP. In other words, if we want to speed up the build, it might be a good idea to build some static base images as a separate action that runs nightly (or via workflow trigger), and for the current branch if-and-only-if the branch changes these dockerfiles.

For example:

• the common section of the Dockerfile
• installing the watcher and/or custom go installations
• apt installs
• static php builds (which means refactoring build-static.sh into a "prepare" and "build" script)

After refactoring, the dockerfiles should be relatively simple: just building FrankenPHP.

These are the acceptance criteria of what I am thinking:

• store base images in the github container registry instead of docker hub
• if there are changes in a PR that affect the base images, they will be built with a -pr#### suffix and the docker bake file will automatically pick that up. This makes it easy to edit without having to know how things work (aka, the Prepare step)
• upon merging or closing a pr, the pr-specific base images should be deleted
• base images should contain everything necessary to build FrankenPHP for the current architecture and only need updating when build dependencies change (such as watcher version, or Go version, etc.).

Since this is a pretty big change to the build infra, I want to run it by everyone before starting on it.

[withinboredom]

I also noticed that there are a lot of dunglas/frankenphp hardcoded throughout the docker files that I'd like to replace with non-hardcoded ones.

[dunglas]

Are custom runners able to leverage the GitHub Actions Cache as hosted runners do?

If yes, there is no need to introduce shared parent images and the like. The Docker Bake action is able to leverage this cache.

I would prefer keeping the Dockerfiles and the build steps as simple as possible. This will help to have reproductible builds (Docker images should be reproductible already, but there is still work to do for static builds).

e-dant/watcher compilation and the like are cacheable, so it shouldn't be an issue.

What we should do however is patch Static PHP CLI to be able to cache downloads and build artifacts. A lock file for the download storing downloaded versions command or something like that could help and would also allow having reproductible builds for static binaries. cc @crazywhalecc

See also crazywhalecc/static-php-cli#187

[withinboredom]

Are custom runners able to leverage the GitHub Actions Cache as hosted runners do?

Yes, but it seems like it may not be as useful as we thought:

This is just from several builds on my fork. In other words, there is probably a lot of random rebuilding depending on how often caches are evicted.

I would prefer keeping the Dockerfiles and the build steps as simple as possible.

Same. This is basically what I'm thinking:

In regular docker commands, this would look something like:

docker build --target common --cache-from $BASE_IMAGE:$GIT_BRANCH --cache-from $BASE_IMAGE:main --cache-to $BASE_IMAGE:$GIT_BRANCH .

Then the build itself would look like the following:

docker build --target runner --tag $IMAGE:$TAG --cache-from $BASE_IMAGE:$GIT_BRANCH --cache-from $CACHE_IMAGE:$GIT_BRANCH --cache-from $CACHE_IMAGE:main --cache-to $CACHE_IMAGE:$GIT_BRANCH

where $BASE_IMAGE is something like ghcr.io/dunglas/frankenphp/caches/base_image and $CACHE_IMAGE is something like ghcr.io/dunglas/frankenphp/caches/runner_cache.

Basically, instead of using action caches, use the container registry itself as a cache.

The actual specifics of tags and such are a bit more involved (for being able to delete them after merging a PR). However, if you were to run it yourself outside of actions, it will "just work"—with a side-effect of some caching images in your local docker.

[crazywhalecc]

The ways that static-php-cli has implemented to speed up the build:

• Provide independent spc binary, which does not require the installation of PHP, Composer and dependencies when building.
• Some libraries provide pre-built compressed packages, use the --prefer-pre-built option.
• For situations where someone don't want to use pre-built packages or the network environment is very bad, they can manually download download-cache.zip.

However, these steps are not very helpful for reproducible builds at present, and static-php is also related to the compilation environment. For example, the latest musl-toolchain built with musl-libc 1.2.5 and gcc-14 cannot compile PHP normally (see crazywhalecc/static-php-cli#537).

It may be feasible to record all system environments and build status into a separate log file. It is not available yet, but it should not be difficult. In addition, regarding the independent setup-static-php project, I am currently busy with my full-time job and do not have much time to fight with js.

[withinboredom]

the latest musl-toolchain built with musl-libc and gcc-14 cannot compile PHP normally

🫠

[withinboredom]

However, these steps are not very helpful for reproducible builds at present, and static-php is also related to the compilation environment.

Luckily, we are compiling in Docker containers, so reproducible has a slightly different meaning for us. We can guarantee the environment won't change (or it will trigger a rebuild) or any other parts of the build chain.

[dunglas]

For the record, we were already using --prefer-pre-built thanks to you @crazywhalecc #921

[AlliBalliBaba]

While we're talking about build speed: Does github offer running actions on ARM hardware? That would massively speed up building the ARM dockers and binary (currently alpine ARM takes 6 hours).

[withinboredom]

Only for enterprise customers.

[dunglas]

According to https://github.blog/changelog/2024-06-03-actions-arm-based-linux-and-windows-runners-are-now-in-public-beta/, they should be available "soon" for open source projects:

We expect to begin offering Arm runners for open source and personal accounts by the end of the year.