Design review for Standard exec configurations

[gregestren]

Hi all.

Standard exec configurations proposes experimenting with patterns for fewer, more clearly defined configurations.

Its immediate focus is to Starlarkize Bazel's exec transition (the configuration compilers and other tools are built in). This opens the door to:

• better access to the exec transition's semantics (currently hard-coded in Bazel and really hard to follow)
• tune the exec transition to reduce unnecessary forks
• let projects swap in custom exec transitions
• remove the need for --host_* flags.

Wider themes are:

• explore the idea of configuration as a group of flags, defined in a BUILD file, referenced by a meaningful name (also Design review for Platform-based Flags #18672 (reply in thread))
• configurations that aren't necessarily patches of other configurations (their definition is "absolute")
• finer criteria for how flag values propagate down the build graph

I originally did a private review with @katre to use Google's repo as a guinea pig testing area. @katre already approved but I wanted to make this more widely visible and to solicit wider feedback.

This is also related to Platform-based flags.

[katre]

My biggest concern with this for Bazel users is how to handle external repositories. If my project Foo depends on rules_bar, and rules_bar defines a new flag --@rules_bar//:bar_version, then the new exec configuration needs to be able to check rules_bar's setup to get the exec value.

Asking every project that uses rules_bar to set the flag in their custom exec transition is the same problem we had with the WORKSPACE file, and will end up with the same solution: rules_bar will publish a bzl file with an exec_transition() function that Foo needs to know to include in Foo's exec transition code. This will lead to the same problem we are solving with bzlmod for recursive dependencies.

We should talk to the bzlmod team and see if there's a good solution there that we can use.

[gregestren]

Great point, and agreed.

Frankly I'm not trying to tackle that yet.

The current phase I'm focusing on is an exact copy of today's built-in semantics, in Starlark. Since that's bundled with Bazel it can't have references Bazel doesn't intrinsically have access to. That should also empower project-custom transitions for anyone interested in applying them. They'd have to ensure their transition definition is comprehensive for the reasons you describe.