Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for nix store remote copying #404

Merged
merged 23 commits into from
Sep 5, 2023
Merged

Conversation

z8v
Copy link
Contributor

@z8v z8v commented Aug 20, 2023

This is a requirement in order to make Bazel remote execution work. Without it it's not guaranteed that any necessary nix paths will be available during the build.

Any local derivation will be copied over to a specified remote nix server which will also as a Bazel executor. This way any nix paths required during the build, will be available.

The user will have to provide the BAZEL_NIX_REMOTE environment variable. This should be the name of any entry on the SSH_CONFIG file where all the authentication details are provided.

e.g

$ export BAZEL_NIX_REMOTE=nix-server

$ cat ~/.ssh/config
Host nix-server
  Hostname 1.2.3.4
  IdentityFile ~/.ssh/nix-server
  Port 2222
  User nix-user

This was done in order to simplify the processes of authentication and keep the number of configuration options to a minimum.

core/nixpkgs.bzl Outdated Show resolved Hide resolved
@z8v
Copy link
Contributor Author

z8v commented Aug 23, 2023

@benradf I believe it would make sense to include in this PR a docker/vagrant setup to test the remote build on the CI?

@benradf
Copy link
Member

benradf commented Aug 23, 2023

That sounds like a great idea actually, though I'm not entirely sure how to go about it.

What were you thinking? Build a docker image as part of CI, run it, and test that it can be used as a remote executor?

@z8v
Copy link
Contributor Author

z8v commented Aug 23, 2023

That sounds like a great idea actually, though I'm not entirely sure how to go about it.

What were you thinking? Build a docker image as part of CI, run it, and test that it can be used as a remote executor?

I was thinking to have a container that will play the role of the nix server. We could run this on the CI so we can test the code path that is enabled when we specify a remote nix server.

@z8v z8v force-pushed the add-remote-exec-support branch 10 times, most recently from 3182de5 to 39e0176 Compare August 28, 2023 12:25
@z8v z8v marked this pull request as ready for review August 28, 2023 12:49
@z8v z8v requested review from benradf and aherrmann August 28, 2023 12:50
Copy link
Member

@benradf benradf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add something like ${{ matrix.withNixRemote && '- NixRemote' || '' }} to the end of the job name in workflow.yaml to distinguish jobs on the withNixRemote dimension.

Otherwise, looks good 👍

@z8v
Copy link
Contributor Author

z8v commented Sep 3, 2023

Maybe add something like ${{ matrix.withNixRemote && '- NixRemote' || '' }} to the end of the job name in workflow.yaml to distinguish jobs on the withNixRemote dimension.

Otherwise, looks good 👍

Updated the title. We'll need to adjust the required checks on Github with the new name.

@z8v z8v requested a review from benradf September 4, 2023 12:51
This is a requirement in order to make Bazel remote execution work. Without it
it's not guaranteed that any necessary nix paths will be available during the
build.

Any local derivation will be copied over to a specified remote nix server which
will also as a Bazel executor. This way any nix paths required during the
build, will be available.

The user will have to provide the `BAZEL_NIX_REMOTE` environment variable. This
should be the name of any entry on the SSH_CONFIG file where all the
authentication details are provided.

e.g

```bash
$ export BAZEL_NIX_REMOTE=nix-server

$ cat ~/.ssh/config
Host nix-server
  Hostname 1.2.3.4
  IdentityFile ~/.ssh/nix-server
  Port 2222
  User nix-user
```

This was done in order to simplify the processes of authentication and keep the
number of configuration options to a minimum.
@benradf benradf added the merge-queue merge on green CI label Sep 5, 2023
@mergify mergify bot merged commit a6a2ea9 into master Sep 5, 2023
15 checks passed
@mergify mergify bot deleted the add-remote-exec-support branch September 5, 2023 15:22
@mergify mergify bot removed the merge-queue merge on green CI label Sep 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants