nixos/xen: refactor dom0 configuration

[SigmaSquadron]

Description of changes

Requires #324693.

• See commit message and release notes.

Things done

• Built on platform(s)
  • x86_64-linux
• Tested, as applicable:
  • No automated tests for NixOS/xen yet.
• 24.11 Release Notes
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
    • I added a release note under highlights. Let me know if this is too presumptuous.
• Fits CONTRIBUTING.md.

Closes #129780, closes #127404.

Add a 👍 reaction to pull requests you find important.

[JulienMalka]

This feels very intrusive to the systemd-boot-builder codebase to me, which will create maintenance burden. I've never worked with Xen, could you explain why this is necessary? Is it not possible to do the modification to the xen.cfg file at build time, like on the file passed to extraFiles?

[SigmaSquadron]

Removed instances of with lib; per review in #324693.

[SigmaSquadron]

I had some time, so I added more options to configure oxenstored.conf. The options haven't changed in a decade, so I think it's alright to skip building a new config generator for oxenstored.conf's custom syntax, and just use etc."xen/oxenstored.conf".text.

xl's next, but that's a major undertaking which is best left for a separate PR.

[CertainLach]

Glad to see Xen is now working in NixOS, can't wait for further progress with Qubes ecosystem here.
Was surprised it suspends correctly on my laptop, unlike QubesOS.

Other than libvirt stuff everything seems to work perfectly fine on couple of test machines.

[SigmaSquadron]

One minor thing that I've noticed: Plymouth is broken when booting in Xen. I doubt it's worth the trouble to investigate, as Xen is mostly used on servers and I don't think desktop users of Xen will care too much about Plymouth.

Still, if anyone has any idea why Plymouth doesn't show up, let me know!

[CertainLach]

Plymouth works for me on intel xe graphics laptop but has a lot more flickering. Probably modesetting/xen video mode related?

[SigmaSquadron]

Hmm, it seems like it doesn't work with the vga=ask dom0 parameter. A shame, since it was the only thing that properly identified my monitor's resolution.

Oddly enough, when booting Xen wasn't working, the parameter would still fix the resolution, and booting into NixOS normally after trying a failing Xen entry would load plymouth with the 2560x1080 resolution instead of the usual 640x480.

[SigmaSquadron]

So, I think this is as ready as it can be, BUT there is one related comment in an unrelated issue that stated that xen-watchdog would sometimes crash dom0 if you updated Xen with a nixos-rebuild switch instead of rebooting. I tried reproducing this by setting my Xen from xenPackages.xen_4_16 to xenPackages.xen_4_19, but xen-watchdog was fine with it. I think it might have been an upstream issue that Xen already fixed, but some testing would be helpful.

[hehongbo]

I can't reproduce it either. I did a quick test on this, starting from 4.16 and then updating it all the way to 4.19 without rebooting.

In my tests, xl command will refuse to work after that, and /sys/hypervisor/version still shows the previous version number, which makes sense, I can't imagine if the underlying hypervisor can be updated when the whole system is running as Dom0 above.

[root@xen-test:~]# xl info
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
host                   : xen-test
release                : 6.6.47
version                : #1-NixOS SMP PREEMPT_DYNAMIC Mon Aug 19 04:04:32 UTC 2024
machine                : x86_64
libxl: error: libxl.c:361:libxl_get_physinfo: getting physinfo: Permission denied
libxl_physinfo failed.
libxl: error: libxl_sched.c:166:libxl_get_scheduler: getting current scheduler id: Permission denied
get_scheduler sysctl failed.
xend_config_format     : 4

[root@xen-test:~]# xl list
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
libxl: error: libxl_utils.c:815:libxl_cpu_bitmap_alloc: failed to retrieve the maximum number of cpus
libxl: error: libxl_domain.c:335:libxl_list_domain: getting domain info list: Permission denied
libxl_list_domain failed.

[root@xen-test:~]# cat /sys/hypervisor/version/major
4

[root@xen-test:~]# cat /sys/hypervisor/version/minor
16

Nothing is failed however, including xen-watchdog.

[root@xen-test:~]# systemctl list-units --failed
  UNIT LOAD ACTIVE SUB DESCRIPTION

0 loaded units listed.

[SigmaSquadron]

I'll consider that issue resolved then.

[SigmaSquadron]

Rebased to resolve merge conflicts in the current master. No functional changes.

[SigmaSquadron]

Once more, just a rebase to fix the merge conflicts in the release notes. Would appreciate a final review/merge before the 24.11 freeze.

[emilazy]

I haven’t done an in‐depth review but it looks basically good to me. A few comments; not sure if I feel qualified to hit the merge button on this unilaterally. (FWIW there’s a month still until feature freeze.)

[emilazy]

Thanks, I like the separate file much more :)

Please ping me if nobody else has given this a proper review in ~a week and I’ll try my best.