Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: extended native-image build options #107

Draft
wants to merge 38 commits into
base: main
Choose a base branch
from

Conversation

sgammon
Copy link
Owner

@sgammon sgammon commented Sep 4, 2023

Summary

Adds more integration points with Bazel's build settings. Initial Platforms stuff might happen in this PR too. Targeting #80 for release if possible (we'll see).

Not yet implemented:
Needs integration with Bazel's Platforms feature, and transition implementations.

Related issues:

Usage

There is a new native_image_settings target, which can be used to specify an explicit bundle of Native Image compilation settings:

load(
  "@rules_graalvm//graalvm:defs.bzl",
  "NativeCompilerBackend",
  "NativeLibC",
  "NativeArchitecture",
  "NativeOptimization",
  "native_image_settings",
)

native_image_settings(
    name = "cool_settings",
    backend = NativeCompilerBackend.LLVM,  # or `NATIVE`
    libc = NativeLibC.GLIBC,  # or `MUSL`
    opt = NativeOptimization.FASTBUILD,  # or `OFF`, `DEFAULT`, or `OPTIMIZATION_LEVEL_[1|2]`
    arch = NativeArchitecture.NATIVE,  # or `COMPATIBILITY`, `AMD_64`, `ARM_64`
)

native_image(
  name = "my_binary",
  # ...
  settings = ":cool_settings",
)

Overriding options via Bazel Configurations

native_image_settings(
    name = "cool_settings",
    arch = NativeImageArchitecture.NATIVE,  # or `COMPATIBILITY`, `AMD_64`, `ARM_64`

    # (`settings` is implied by default, pass `None` to opt-out)
    settings = "@rules_graalvm//graalvm/native_image:defaults",
)

If you do nothing at all, or map the settings property this way (or provide your own equivalent native_image_info advice target), then certain flags will override your native_image_settings. With the above configuration, settings are effectively resolved from the following places, in order:

1) Overrides provided by the command line, like below
2) Explicit settings declared in the mapped `native_image_settings` bundle
3) Sensible defaults provided by `rules_graalvm` (for example, `compilation_mode` → `O[x]`)

This allows you to set a default march, for instance:

native_image_settings(
  name = "settings",
  arch = NativeArchitecture.COMPATIBILITY,
)

native_image(
  # ...
  settings = ":settings",
)

Which you can override easily on the command line when desired:

bazel build //your/native:target --@rules_graalvm//graalvm/nativeimage/options:arch=native

How you know it's working

In the native-image build output, the target machine will be set accordingly:

bazel build //your/native:target --@rules_graalvm//graalvm/nativeimage/options:arch=native
========================================================================================================================
GraalVM Native Image: Generating 'main-native-bin' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------
[1/8] Initializing...                                                                                    (2.7s @ 0.17GB)
 Java version: 20.0.2+9, vendor version: GraalVM CE 20.0.2+9.1
 Graal compiler: optimization level: b, target machine: native

...

Opting out of rules-controlled flags

To remove (some) of the Rules-controlled flags and instead withhold them or provide them via extra_args, you can pass None to the settings attribute, which prevents override:

native_image_settings(
    name = "minimal",
    settings = None,
)

native_image(
  # ...
  settings = ":minimal",
)

In this case, these flags are withheld from automatic injection, although you may still need to provide some of them to complete a native-image build. Effectively, most flags take on GraalVM's defaults.

Changelog

  • feat: add experimental build setting which controls the march parameter

fmeum and others added 30 commits September 3, 2023 17:32
Instead of relying on the default shell env, get the environment
variables declared by the C++ toolchain. On macOS, additionally use
`apple_support` to pass in `DEVELOPER_DIR`.

Since GraalVM sanitizes the environment, all variables are translated
into `-E` flags.

Signed-off-by: Sam Gammon <sam@elide.ventures>
- fix: move `lambda` to nested function declaration

Signed-off-by: Sam Gammon <sam@elide.ventures>
- fix: unwind combined implementation of classic/modern rules
- fix: classic rules should preserve older functionality
- fix: modern rules should use modern toolchain env
- fix: wrapped/injected `env` while still supporting bazel4
- chore: re-factor commons between old/new rules

Signed-off-by: Sam Gammon <sam@elide.ventures>
- fix: use legacy rules from legacy gvm
- chore: drop `MODULE-resolved.bzl`
- chore: update bzlmod lock
- chore: update lib/docs deps and rebuild docs
- chore: remove redundant calls in sample projects

Applied on top of #72

Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
When provided, the static zlib library will be used when Graal
statically links in zlib.

Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Fabian Meumertzheim <fabian@meumertzhe.im>
Signed-off-by: Sam Gammon <sam@elide.ventures>
- fix: add `gvm` toolchain to toolchain config repo
- fix: add aliases from `graalvm` → toolchain targets
- fix: adjust instructions for registering toolchains
- fix: adjust workspace toolchain registration logic

Breaking change:
When registering toolchains in a Bzlmod installation of these rules,
the target `@graalvm//:all` must be changed to two toolchain
registrations, based on the desired functionality:

  Register the Java toolchain:
  register_toolchains("@graalvm//:jvm")

  Register the GVM toolchain:
  register_toolchains("@graalvm//:sdk")

Fixes and closes #66.

Relates-To: #66
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
- test: add `java-toolchain` test, which exercises graalvm as the
  bazel tool/runtime java toolchain.

- test: add `components-ce` test, which downloads GVM Community
  components with dependencies.

- test: add `components-gvm` test, which downloads Oracle GVM
  components with `gu`.

- test: add `disabled_tests` with tests for earlier versions of
  bazel (coming soon), including `bazel1`, `bazel2`, and `bazel3`

- test: move most tests to GVM Community (except where noted)

Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
- feat: detect and honor `compilation_mode` flag
- feat: build with debug settings if `compilation_mode=dbg`
- feat: ability to build a shared library
- feat: build with `tool:coverage` if coverage is enabled and a
  test-only target is being built
- docs: add doc which explains shared library feature
- docs: add doc for built settings integration (more to come)

Relates-To: #78
Relates-To: #85
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
renovate bot and others added 7 commits September 3, 2023 17:32
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
- test: add code to `components-ce` test to ensure `js`
  component is installed and usable

Co-authored-by: Mantas Indrašius <mantasi@wix.com>
Signed-off-by: Sam Gammon <sam@elide.ventures>
Signed-off-by: Sam Gammon <sam@elide.ventures>
@sgammon sgammon added feature Mainline feature work native-image Features and issues relating to the Native Image tool 🚧 WIP Work-in-progress, do not merge labels Sep 4, 2023
@sgammon sgammon added this to the 1.0.0 milestone Sep 4, 2023
@sgammon sgammon self-assigned this Sep 4, 2023
@sgammon sgammon mentioned this pull request Sep 2, 2023
8 tasks
- feat: add experimental build setting which controls the `march`
  parameter

- feat: explicit settings with `native_image_settings`, flag
  overrides with `native_image_info`

- fix: preserve legacy rules `march` flag (missing by default)

Not yet implemented:
Needs integration with Bazel's platforms feature.

Signed-off-by: Sam Gammon <sam@elide.ventures>
@sonarcloud
Copy link

sonarcloud bot commented Sep 4, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@CLAassistant
Copy link

CLAassistant commented Nov 30, 2023

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

❌ fmeum
❌ renovate[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

@sgammon sgammon changed the base branch from release/0.10.x to release/0.11.0 January 5, 2024 03:46
@sgammon sgammon changed the base branch from release/0.11.0 to main January 5, 2024 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Mainline feature work native-image Features and issues relating to the Native Image tool 🚧 WIP Work-in-progress, do not merge
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

3 participants