Multiarch setups with QEMU

Multi-arch setups with QEMU

By default Docker can only run images that have the same architecture as the host. Most widespread are amd64 images running on amd64 hosts. With a regular setup it is not possible to run e.g. images with an arm/aarch64 architecture on amd64.

• You can check the image architecture with docker inspect --format {{.Architecture}} IMAGENAME.
• You can check the host architecture with uname -m.

However, with some setup it is possible to run arm/aarch64 images on amd64 and vice versa. Same goes for some other architectures. This is out of scope for x11docker that just want to provide an X server and some extra features, but of interest for multi-arch GUI development with the help of x11docker. x11docker itself does not care for the architecture, this is up to Docker and its configuration. List of architectures supported by Docker.

Possibilities:

• Use QEMU to run images with a different architecture than the host.
• Use buildx to build images targeting different architectures.

QEMU

QEMU allows executing foreign architectures through dynamic translation of the instructions. Combining QEMU and Docker, it is possible to execute containers for e.g. ARM on AMD64, or vice versa. Find detailed explanations, utilities and examples at dbhi/qus. Moreover, as a reference, in dbhi/docker images from qus are used to build and publish multi-arch images/manifests in an AMD64 only CI service.

A shortcut to allow multiple archtitectures without installing QEMU on host is docker run --rm --privileged aptman/qus -s -- -p. This container adds some QEMU features to the host kernel. Technically, it uses a shell script to register binfmt interpreters based on the magic strings from QEMU, and point them to QEMU user static binaries. After executing that command, any foreign binary can be used on the host as long as it is statically built, or foreign dependencies exist. Precisely, using foreign containers avoids installing cross-platform toolchains.

It is mostly meant for CI and for development purposes. On the one hand, using this approach allows to build, test and publish multi-arch images/manifests using regular amd64 only CI services (such as GitHub Actions or Travis CI). On the other hand, it allows developing and testing applications for embedded devices (say SBCs or SoCs with FPGA) from amd64 workstations.

For example, binaries built and tested in arm32v7/ubuntu containers can be deployed to PYNQ boards (https://github.com/Xilinx/PYNQ/releases). Such embedded devices are typically slower than workstations, specially for cpu-bound tasks. Hence, even if QEMU introduces a x3-10 execution penalty, development can be faster. Furthermore, using a container is "cleaner" than installing all cross-compilation dependencies on the host. Last, but not least, some of those boards cost >2000€. Hence, the number of devices in the lab (or at home) might be limited.

buildx

With buildx the same Dockerfile can be used to produce in parallel images which can execute on different hardware architecture. buildx requires an installation of QEMU and some experimental features of Docker build. The result is a set of comparable Docker images, which run on different hardware and produce identical effects. An introduction is given in Artur Klauser: Building Multi-Architecture Docker Images With Buildx. Basically buildx allows to automate multiple architecture builds and manifest generation in one command.