nixos/netbird: harden and extend options #287236

nazarewk · 2024-02-08T12:25:38Z

Description of changes

I have recently extensively tested and fixed all features of Netbird in my own implementation of multi-instance Netbird installations.

While doing so I discovered another multi-instance implementation got merged into nixpkgs #246055 which is slightly different, but still a solid base to upstream the rest of my changes:

running as ~~DynamicUser~~ it's own user with minimal set of permissions
- it was there before, but was lacking some of capabilities,
made some configurations situational
add more unmanaged interface configurations
quality of life improvements:
- configure log level for each interface
- optionally turn off starting during boot
- openFirewall by default
- add shortcuts/wrappers for each created instance

I think it's a pretty good time to upstream, because I will be extensively using it at work: just launched my first Colmena-managed NixOS into GCE.

There are plans to support multi-account connections on the same daemon in Q2/2024 (see the slack message), but it's not known what shape it will take at all.

I decided to implement following significant changes:

instances must specify a port they will be listening on as it doesn't make much sense to give an immediately conflicting default,
aliased tunnels to clients, because a word tunnel does not exist in Netbird's nomenclature (unlike some other VPNs) and is pretty misleading. Also clients.* play nicely with my plan to implement a server in near future.
skipped destructuring expressions (eg: {name, ...}: name -> client: client.name) because they make the code very hard to follow and update with increased number of options,

Things done

Add a 👍 reaction to pull requests you find important.

Signed-off-by: Krzysztof Nazarewski <gpg@kdn.im>

nixos/modules/services/networking/netbird.nix

Tom-Hubrecht · 2024-02-09T16:59:21Z

nixos/modules/services/networking/netbird.nix

+{ config
+, lib
+, pkgs
+, ...


Reformatting this should not be commited

this is what nixpkgs-fmt does, as far as i remember the code should be formatted with it?

nixos/modules/services/networking/netbird.nix

Tom-Hubrecht · 2024-02-09T17:09:22Z

nixos/modules/services/networking/netbird.nix

+              suffix = mkOption {
+                type = str;
+                default = "-${name}";
+                description = ''
+                  Suffix to use for this further naming of the service pieces.
+                '';
+              };


This seems overly complicated, as it is only used for defaults, you should use the name for this

It is also used for the wrapper, but yeah might make more sense to not drop -wt0 from the default config. The raw "unwrapped" package should work with it anyway.

it made more sense when there were 3 wrappers

nixos/modules/services/networking/netbird.md

Tom-Hubrecht · 2024-02-12T22:11:20Z

nixos/tests/netbird.nix

@@ -12,6 +12,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }:
    };
  };

+  # TODO: confirm DynamicUser is working if/after netbird server is implemented


There is no netbird server in this PR

yeah, but it is very much relevant for the test, there is no confirmation the client actually works until we have a server to connect to.

Tom-Hubrecht · 2024-02-12T22:25:23Z

nixos/modules/services/networking/netbird.nix

+                    exports = pipe config.environment [
+                      (mapAttrsToList (name: default:
+                        # Couldn't default with `''${${name}:${escapeShellArg default}}`
+                        # because `escapeShellArg` wraps values in single quotes,
+                        # which in turn works differenty (or rather doesn't) than double-quotes
+                        # in this shell parameter expansion
+                        ''test "''${${name}+X}" == X || export ${toShellVar name default}''))
+                      (concatStringsSep "\n")
+                    ];
+                  in
+                  pkgs.writeShellScriptBin "netbird${config.suffix}" ''
+                    ${exports}
+                    ${lib.getExe' cfg.package "netbird"} "$@"
+                  '';


This is very weird, nixpkgs has a machinery already to wrap programs c.f. https://nixos.org/manual/nixpkgs/stable/#fun-wrapProgram

I'm pretty sure it would be significantly longer with wrapProgram, something like:

mkDerivation { pname = "xxx"; installPhase = let exports = pipe config.environment [ (mapAttrsToList (name: default: "--set-default '${name}' '${default}'")) (concatStringsSep " ") ]; in "wrapProgram ${lib.getExe' cfg.package "netbird"} $out/bin/netbird${config.suffix} ${exports}"; }

actually makes more sense after prototyping it and seeing with own eyes

You can do something like

stdenv.mkDerivation { intallPhase = '' wrapProgram ${lib.getExe cfg.package} $out/bin/netbird${config.suffix} ${ concatMapStringsSep " " ({ name, value }: "--set-default '${name}' '${default}'") ( attrsToList config.environment ) } '';

The issue I had was not because of the size of the code, but reinventing the wheel ^^

not so short anymore:

let makeWrapperArgs = flatten (mapAttrsToList (key: value: [ "--set-default" key value ]) config.environment ); in pkgs.stdenv.mkDerivation { name = "${cfg.package.name}-wrapper-${name}"; meta.mainProgram = config.name; nativeBuildInputs = with pkgs; [ makeWrapper ]; phases = [ "installPhase" ]; installPhase = '' mkdir -p "$out/bin" makeWrapper ${lib.getExe cfg.package} "$out/bin/${config.name}" \ ${escapeShellArgs makeWrapperArgs} ''; }

Tom-Hubrecht · 2024-02-12T22:26:30Z

nixos/modules/services/networking/netbird.nix

+
+        A single daemon is fully operational.
+
+        Multiple daemons will interfere with each other's DNS resolution, but otherwise should remain operational.


When the networks have different domains, there is no interference, the only issue is when then share the same domain

I don't think the domain is configurable at all on the Netbird server side?

It is when you self-host your server

FYI:

Hello[@nazarewk, we plan a custom DNS zone support coming next quarter. That would allow you to create custom domains for applications and peers

https://netbirdio.slack.com/archives/C02KHAE8VLZ/p1707820381270479?thread_ts=1707819591.188209&cid=C02KHAE8VLZ

Custom DNS zone support is already implemented for self-hosted,

${pkgs.netbird}/bin/netbird-mgmt management \ --dns-domain dns.domain

Will place every machine under dns.domain subdomain: machine1.dns.domain, machine2.dns.domain

nixos/modules/services/networking/netbird.nix

oddlama · 2024-06-02T12:13:44Z

nixos/modules/services/networking/netbird.nix

+              name = mkOption {
+                type = str;
+                default = name;
+                apply = value: "netbird-${value}";


I think it would be better style to force the prefix in the relevant locations, otherwise /var/lib/${cfg.name} will always look problematic since it implies any name is possible.

sounds reasonable

oddlama

I think you'll have to adjust the release notes to apply to 24.11 now that 24.05 is out the door. But the important details look fine to me, so this is good to go from my side. We can iron out any possible details separately if necessary.

oddlama · 2024-06-09T12:13:57Z

There are still changes to nixos/doc/manual/release-notes/rl-2405.section.md

nazarewk · 2024-06-09T17:25:46Z

There are still changes to nixos/doc/manual/release-notes/rl-2405.section.md

Yeah, the nixos manual command refused to pass, assumed it was expecting me to change it?

oddlama · 2024-06-09T17:53:55Z

Yeah, the nixos manual command refused to pass, assumed it was expecting me to change it?

I guess it complains because you cannot edit old release notes

nazarewk · 2024-06-10T09:17:08Z

Yeah, the nixos manual command refused to pass, assumed it was expecting me to change it?

I guess it complains because you cannot edit old release notes

it refused to pass without changes https://github.com/NixOS/nixpkgs/actions/runs/9399057023/job/25885722356

github-actions bot added 6.topic: nixos 8.has: module (update) labels Feb 8, 2024

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Feb 8, 2024

nazarewk changed the title ~~nixos/netbird: run as DynamicUser with more configuration options~~ nixos/netbird: bring back DynamicUser with more configuration options Feb 9, 2024

nazarewk marked this pull request as draft February 9, 2024 08:43

nazarewk changed the title ~~nixos/netbird: bring back DynamicUser with more configuration options~~ nixos/netbird: harden and extend options Feb 9, 2024

nazarewk force-pushed the netbird-improvements branch 5 times, most recently from 0cf761f to 4179661 Compare February 9, 2024 12:47

nazarewk marked this pull request as ready for review February 9, 2024 12:54

nazarewk added a commit to nazarewk-iac/nix-configs that referenced this pull request Feb 9, 2024

nixos/netbird: switch to NixOS/nixpkgs#287236

eaba33c

Signed-off-by: Krzysztof Nazarewski <gpg@kdn.im>

Tom-Hubrecht assigned mlvzk and unassigned mlvzk Feb 10, 2024

nazarewk requested review from misuzu and Tom-Hubrecht February 12, 2024 07:43

misuzu suggested changes Feb 12, 2024

View reviewed changes

nazarewk mentioned this pull request Feb 12, 2024

NB_{WIREGUARD_PORT,INTERFACE_NAME} is not supported by anything except netbird up netbirdio/netbird#1562

Open

nazarewk force-pushed the netbird-improvements branch from 4179661 to 0a1d920 Compare February 12, 2024 15:30

nazarewk requested a review from misuzu February 12, 2024 15:33

nazarewk force-pushed the netbird-improvements branch from 0a1d920 to add2bf7 Compare February 12, 2024 15:46

github-actions bot added 8.has: documentation 8.has: changelog labels Feb 12, 2024

nazarewk force-pushed the netbird-improvements branch from add2bf7 to 26373ef Compare February 12, 2024 20:39

Tom-Hubrecht suggested changes Feb 12, 2024

View reviewed changes

nazarewk force-pushed the netbird-improvements branch 3 times, most recently from a734f1e to 35a7c67 Compare February 13, 2024 11:08

ofborg bot removed the 2.status: merge conflict label May 25, 2024

oddlama reviewed Jun 2, 2024

View reviewed changes

nazarewk force-pushed the netbird-improvements branch 2 times, most recently from de1630c to f3047d3 Compare June 4, 2024 09:24

oddlama reviewed Jun 4, 2024

View reviewed changes

nazarewk force-pushed the netbird-improvements branch 2 times, most recently from 7e67c54 to 89b5ecf Compare June 4, 2024 11:09

ofborg bot added the 2.status: merge conflict label Jun 4, 2024

nazarewk requested a review from oddlama June 6, 2024 10:11

nazarewk force-pushed the netbird-improvements branch from 89b5ecf to e8588b2 Compare June 6, 2024 10:11

ofborg bot removed the 2.status: merge conflict label Jun 6, 2024

nazarewk force-pushed the netbird-improvements branch from e8588b2 to 5b5f809 Compare June 6, 2024 14:07

nazarewk force-pushed the netbird-improvements branch from 5b5f809 to 8356170 Compare June 24, 2024 08:46

wegank added the 2.status: merge conflict label Jul 4, 2024

nazarewk force-pushed the netbird-improvements branch from 8356170 to 79b769d Compare July 15, 2024 11:44

nazarewk force-pushed the netbird-improvements branch from 79b769d to 2de52c1 Compare July 30, 2024 10:37

nazarewk force-pushed the netbird-improvements branch from 2de52c1 to 72e61ff Compare September 2, 2024 18:34

nazarewk mentioned this pull request Sep 17, 2024

netbird: remove misuzu as maintainer #341812

Merged

13 tasks

nazarewk force-pushed the netbird-improvements branch 2 times, most recently from 6947525 to 41ea15c Compare September 19, 2024 13:19

ofborg bot removed the 2.status: merge conflict label Sep 19, 2024

nazarewk force-pushed the netbird-improvements branch from 41ea15c to 5681331 Compare September 26, 2024 08:34

netbird-ui: fix incorrect meta.mainProgram

99f838e

nazarewk force-pushed the netbird-improvements branch from 5681331 to b11a6de Compare September 26, 2024 08:36

nixos/netbird: harden and extend options

43d2daf

nazarewk force-pushed the netbird-improvements branch from b11a6de to 43d2daf Compare September 26, 2024 08:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/netbird: harden and extend options #287236

nixos/netbird: harden and extend options #287236

nazarewk commented Feb 8, 2024 •

edited

Loading

Tom-Hubrecht Feb 9, 2024

nazarewk Feb 13, 2024

Tom-Hubrecht Feb 9, 2024

nazarewk Feb 13, 2024

nazarewk Feb 13, 2024 •

edited

Loading

Tom-Hubrecht Feb 12, 2024

nazarewk Feb 13, 2024 •

edited

Loading

Tom-Hubrecht Feb 12, 2024

nazarewk Feb 13, 2024

Tom-Hubrecht Feb 13, 2024

nazarewk Feb 13, 2024

Tom-Hubrecht Feb 12, 2024

nazarewk Feb 13, 2024

Tom-Hubrecht Feb 13, 2024

nazarewk Feb 13, 2024

CertainLach Jun 6, 2024

oddlama Jun 2, 2024

nazarewk Jun 4, 2024

oddlama left a comment

oddlama commented Jun 9, 2024

nazarewk commented Jun 9, 2024

oddlama commented Jun 9, 2024

nazarewk commented Jun 10, 2024


		A single daemon is fully operational.

		Multiple daemons will interfere with each other's DNS resolution, but otherwise should remain operational.

nixos/netbird: harden and extend options #287236

Are you sure you want to change the base?

nixos/netbird: harden and extend options #287236

Conversation

nazarewk commented Feb 8, 2024 • edited Loading

Description of changes

Things done

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nazarewk Feb 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nazarewk Feb 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oddlama left a comment

Choose a reason for hiding this comment

oddlama commented Jun 9, 2024

nazarewk commented Jun 9, 2024

oddlama commented Jun 9, 2024

nazarewk commented Jun 10, 2024

nazarewk commented Feb 8, 2024 •

edited

Loading

nazarewk Feb 13, 2024 •

edited

Loading

nazarewk Feb 13, 2024 •

edited

Loading