Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add fast multi-platform build #389

Merged
merged 28 commits into from
Nov 19, 2024
Merged

feat: add fast multi-platform build #389

merged 28 commits into from
Nov 19, 2024

Conversation

merklefruit
Copy link
Collaborator

@merklefruit merklefruit commented Nov 15, 2024

This PR adds support for building cross-platform binaries for x86_64 and aarch64 (unknown-linux-gnu).
We use cargo cross to produce a separate release binary for each architecture and successively load it
into a docker image directly.

Benefits:

  • builds are fast, the part that takes longer is the linking at the end.
  • no Docker overhead for build contexts, meaning also no need for janky docker cache layers
  • lightweight output images, because we only ship the binary inside them

Downsides:

  • since the images only contain the binary, it's harder to verify that they are running the correct binary. This is not a big issue because anyone wanting to verify their image integrity can build it locally from source.

Usage

Available packages that can be built from this repo (so far): bolt-sidecar, bolt-boost.

Example to produce a cross-platform image with tag unstable:

just build-and-push-image bolt-sidecar unstable

Example to produce cross-platform images for all available packages mentioned above with tag unstable:

just build-and-push-all-images unstable

Requisites to build images

  • cargo
  • cross
  • docker buildx

@merklefruit merklefruit added the A: infra Area: Infrastructure label Nov 15, 2024
@merklefruit merklefruit self-assigned this Nov 15, 2024
Copy link
Contributor

@thedevbirb thedevbirb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, super happy we got it working with the binary-based approach

@merklefruit merklefruit marked this pull request as draft November 18, 2024 09:38
@merklefruit
Copy link
Collaborator Author

Converted to draft because there is an openssl issue when building bolt-boost for aarch64

ssz_rs = { git = "https://github.com/ralexstokes/ssz-rs", rev = "ec3073e", features = [
"sha2-asm",
] }
ssz_rs = { git = "https://github.com/ralexstokes/ssz-rs", rev = "ec3073e" }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to remove the ssz-rs sha2-asm feature as it was breaking for cross-compiled aarch64.
There might be a way to do it but for now I had to disable it

@merklefruit
Copy link
Collaborator Author

Images tagged v0.3.0-alpha have been produced from the latest commit.

@merklefruit merklefruit marked this pull request as ready for review November 18, 2024 21:09
Copy link
Contributor

@thedevbirb thedevbirb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🫡

@thedevbirb thedevbirb merged commit e2d78bd into unstable Nov 19, 2024
3 checks passed
@thedevbirb thedevbirb deleted the nico/cross-builds branch November 19, 2024 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: infra Area: Infrastructure
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants