muslrust

A docker environment for building static rust binaries for x86_64 and arm64 linux environments using musl. Built daily via github actions.

Binaries compiled with muslrust are light-weight, call straight into the kernel without other dynamic system library dependencies, can be shipped to most linux distributions without compatibility issues, and can be inserted as-is into lightweight docker images such as static distroless, scratch, or alpine.

The goal is to simplify the creation of small and efficient cloud containers, or stand-alone linux binary releases.

This image includes popular C libraries compiled with musl-gcc, enabling static builds even when these libraries are used.

Usage

Pull and run from a rust project root:

docker pull clux/muslrust:stable
docker run -v $PWD:/volume --rm -t clux/muslrust:stable cargo build --release

You should have a static executable in the target folder:

ldd target/x86_64-unknown-linux-musl/release/EXECUTABLE
        not a dynamic executable

Examples

• Kubernetes controller with actix-web using plain distroless/static
• Kubernetes reflector with axum using builder pattern
• Kubernetes controller using cargo-chef for caching layers
• Github release assets uploaded via github actions
• Using muslrust with sccache & github actions

The binaries and images for small apps generally end up <10MB compressed or ~20MB uncompressed without stripping.

The recommended production image is distroless static or chainguard static as these contain a non-root users + SSL certs (unlike scratch), and disallows shell access (use kubectl debug if you want this). See also kube.rs security doc on base image recommendations.

Available Tags

The standard tags are :stable or a dated :nightly-{YYYY-mm-dd}.

For pinned, or historical builds, see the available tags on dockerhub.

C Libraries

The following system libraries are compiled against musl-gcc:

• curl (curl crate)
• openssl (openssl crate)
• pq (pq-sys crate used by diesel)
• sqlite3 (libsqlite3-sys crate used by diesel)
• zlib (used by pq and openssl)

We try to keep these up to date.

Developing

Clone, tweak, build, and run tests:

git clone git@github.com:clux/muslrust.git && cd muslrust
just build
just test

Tests

Before we push a new version of muslrust we test to ensure that we can use and statically link:

• serde
• diesel
• hyper
• curl
• rustls
• openssl
• flate2
• rand

Caching

Local Volume Caches

Repeat builds locally are always from scratch (thus slow) without a cached cargo directory. You can set up a docker volume by just adding -v cargo-cache:/root/.cargo/registry to the docker run command.

You'll have an extra volume that you can inspect with docker volume inspect cargo-cache.

Suggested developer usage is to add the following function to your ~/.bashrc:

musl-build() {
  docker run \
    -v cargo-cache:/root/.cargo/registry \
    -v "$PWD:/volume" \
    --rm -it clux/muslrust cargo build --release
}

Then use in your project:

$ cd myproject
$ musl-build
    Finished release [optimized] target(s) in 0.0 secs

Caching on CI

On CI, you need to find a way to either store the cargo-cache referenced above, or rely on docker layer caches with layers (see cargo-chef).

Github Actions

Github actions supports both methods:

• GHA: direct folder cache (manual docker build)
• GHA: via docker layer caches (builder-pattern) (with cargo-chef)

CircleCI

CircleCI supports both methods:

• Circle: direct folder cache (manual docker build)
• Circle also supports docker layer caching (no example atm)

Allocator Performance

To optimise memory performance (see #142) consider changing the global allocators in sensitive applications:

• jemalloc
• mimalloc

use tikv_jemallocator::Jemalloc;
#[global_allocator]
static GLOBAL: Jemalloc = Jemalloc;

Troubleshooting

SSL Verification

You might need to point openssl at the location of your certificates explicitly to avoid certificate errors on https requests.

export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
export SSL_CERT_DIR=/etc/ssl/certs

These can be hardcoded in your Dockerfile, or you can rely on the openssl-probe crate to detect the cert location. You should not have to do this if you are using the static variants of distroless or chainguard.

Diesel and PQ builds

Works with the older version of libpq we bundle (see #81). See the test/dieselpgcrate for specifics.

For stuff like infer_schema! to work you need to explicitly pass on -e DATABASE_URL=$DATABASE_URL to the docker run. It's probably easier to just make diesel print-schema > src/schema.rs part of your migration setup though.

Note that diesel compiles with openssl statically since 1.34.0, so you need to include the openssl crate before diesel due to pq-sys#25:

extern crate openssl;
#[macro_use] extern crate diesel;

This is true even if you connect without sslmode=require.

Filesystem permissions on local builds

When building locally, the permissions of the musl parts of the ./target artifacts dir will be owned by root and requires sudo rm -rf target/ to clear. This is an intended complexity tradeoff with user builds.

Debugging in blank containers

If you are running a plain alpine/scratch container with your musl binary in there, then you might need to compile with debug symbols, and set the RUST_BACKTRACE=full evar to see crashes.

In alpine, if this doesn't work (or fails to give you line numbers), try installing the rust package (via apk). This should not be necessary anymore though!

For easily grabbing backtraces from rust docker apps; try adding sentry. It seems to be able to grab backtraces regardless of compile options/evars.

SELinux

On SELinux enabled systems like Fedora, you will need to configure selinux labels. E.g. adding the :Z or :z flags where appropriate: -v $PWD:/volume:Z.

Extending

Extra C libraries

If you need extra C libraries, you can inherit from this image FROM clux/muslrust:stable as builder and add extra curl -> make instructions. We are unlikely to include other C libraries herein unless they are very popular.

Extra Rustup components

You can install extra components distributed via Rustup like normal:

rustup component add clippy

Binaries distributed via Cargo

If you need to install a binary crate such as ripgrep on a CI build image, you need to build it against the GNU toolchain (see #37):

CARGO_BUILD_TARGET=x86_64-unknown-linux-gnu cargo install ripgrep

Alternatives

• rustup target add x86_64-unknown-linux-musl works locally when not needing C libraries
• official rust image can target add and easily cross-build when not needing C libraries
• cross can cross-build different embedded targets