Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add fast multi-platform build #389

Merged
merged 28 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0b26e97
feat: use cross for multiplatform builds
merklefruit Nov 15, 2024
68f11f0
feat: zigbuild
merklefruit Nov 15, 2024
f1b55b9
fix: build-arg
merklefruit Nov 15, 2024
29aa316
fix: target path
merklefruit Nov 15, 2024
c0f6881
chore: fixes
merklefruit Nov 15, 2024
2a0accc
feat: use base dir to hold binaries
merklefruit Nov 15, 2024
0671f11
fix: adjust copy path in cross build file
merklefruit Nov 15, 2024
93117cc
fix: use static name for binary output file
merklefruit Nov 15, 2024
7211b15
fix: attempt at native cargo on native arch
merklefruit Nov 15, 2024
738f7f8
chore: rm shell
merklefruit Nov 15, 2024
c64c253
chore: debug logs
merklefruit Nov 15, 2024
4697777
chore: handle edge case with cargo
merklefruit Nov 15, 2024
6c97f61
chore: typo
merklefruit Nov 15, 2024
9e3704c
feat: add cross compiled openssl path
merklefruit Nov 18, 2024
da19682
chore: fix cross compile script
merklefruit Nov 18, 2024
556468d
chore: adjust openssl paths
merklefruit Nov 18, 2024
4519cbc
fix: more fixes to cross build script
merklefruit Nov 18, 2024
97255ca
fix: use correct compiler
merklefruit Nov 18, 2024
f71fa98
chore: more cross compilation tests
merklefruit Nov 18, 2024
966de9e
chore: more cross compilation tests
merklefruit Nov 18, 2024
ec349f7
chore: use cross for bolt-sidecar
merklefruit Nov 18, 2024
2d3d7b5
chore: typo
merklefruit Nov 18, 2024
38acf9f
chore: typo
merklefruit Nov 18, 2024
e019f4f
chore: only arm64 with cross
merklefruit Nov 18, 2024
4df28d6
feat: use cross for everything with openssl cross compiled lib
merklefruit Nov 18, 2024
3c88a0e
fix: cargo clean between builds, use more recent gcc
merklefruit Nov 18, 2024
3e565c1
chore: rm compilation support script, update docs
merklefruit Nov 18, 2024
c9fc73f
chore: add troubleshooting tips
merklefruit Nov 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
# log files
*_dump.log
logs/
target/

# environment secrets
.env

# IDEs
.vscode
.idea

# build artifacts
target/
dist/

# other
.DS_Store
.env
56 changes: 45 additions & 11 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -136,23 +136,57 @@ send-blob-preconf count='1':
--count {{count}}

# build all the docker images locally
build-images:
@just build-sidecar
@just build-bolt-boost
build-local-images:
@just build-local-sidecar
@just build-local-bolt-boost

# build the docker image for the bolt sidecar
[private]
build-sidecar:
build-local-sidecar:
cd bolt-sidecar && docker build -t ghcr.io/chainbound/bolt-sidecar:0.1.0 . --load

# build the docker image for bolt-boost
[private]
build-bolt-boost:
build-local-bolt-boost:
cd bolt-boost && docker build -t ghcr.io/chainbound/bolt-boost:0.1.0 . --load

# build and push multi-platform docker images to GHCR with the provided tag
[confirm("are you sure? this will build and push new images on ghcr.io")]
release tag:
chmod +x ./scripts/check_version_bumps.sh && ./scripts/check_version_bumps.sh {{tag}}
cd bolt-sidecar && docker buildx build --platform linux/amd64,linux/arm64 -t ghcr.io/chainbound/bolt-sidecar:{{tag}} --push .
cd bolt-boost && docker buildx build --platform linux/amd64,linux/arm64 -t ghcr.io/chainbound/bolt-boost:{{tag}} --push .

# Cross platform builds
#
# We use cargo-zigbuild to build cross-platform binaries faster and to get around
# weird linking bugs that often happen with cross or similar tools.
#
# Steps:
# 1. install zig:
# ubuntu: snap install zig --classic --beta
# macOS: brew install zig
# others: https://github.com/ziglang/zig/wiki/Install-Zig-from-a-Package-Manager
# 2. install zigbuild: cargo install cargo-zigbuild
#
# build the cross platform binaries for a package by name. available: "bolt-sidecar", "bolt-boost".
[private]
cross-build-binary package target_arch release_dir:
rustup target add {{target_arch}}


cd {{package}} && cargo zigbuild --target {{target_arch}} --profile release

mkdir -p dist/bin/{{release_dir}}
cp {{package}}/target/{{target_arch}}/release/{{package}} dist/bin/{{release_dir}}/{{package}}

# build and push multi-platform docker images to GHCR for a package. available: "bolt-sidecar", "bolt-boost".
build-and-push-image package tag:
@just cross-build-binary {{package}} x86_64-unknown-linux-gnu amd64
@just cross-build-binary {{package}} aarch64-unknown-linux-gnu arm64

docker buildx build \
--build-arg BINARY={{package}} \
--file ./scripts/cross.Dockerfile \
--platform linux/amd64,linux/arm64 \
--tag ghcr.io/chainbound/{{package}}:{{tag}} \
--push .

# build and push all the available packages to GHCR with the provided tag
build-and-push-all-images tag:
@just build-and-push-image bolt-sidecar {{tag}}
@just build-and-push-image bolt-boost {{tag}}
18 changes: 18 additions & 0 deletions scripts/cross.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# This image is meant to enable cross-architecture builds.
# It assumes the binary has already been compiled for `$TARGETPLATFORM` and is
# locatable in `./dist/bin/$TARGETARCH/$BINARY`.

FROM --platform=$TARGETPLATFORM ubuntu:22.04

LABEL org.opencontainers.image.source=https://github.com/chainbound/bolt
LABEL org.opencontainers.image.licenses="MIT"

# Filled by docker buildx
ARG TARGETARCH

# Should be set by the caller when building the image
ARG BINARY

COPY ./dist/bin/$TARGETARCH/$BINARY /usr/local/bin/bolt

ENTRYPOINT ["/usr/local/bin/bolt"]