Define gccFromSrc to allow building gcc from a source

[silvanshade]

Description of changes

This PR defines a pkgs.gccFromSrc function which uses gccFun and wrapCC to allow building a custom gcc package from a src, such as a local directory or a GitHub repository.

It also updates some parts of the gcc definitions slightly to account for upcoming version 14. Basically the only real change necessary is that flex needs to be provided as an input.

With this change, it's possible to define custom GCC 14 as in the following example. Below, we define pkgs.gcc14 and pkgs.pkgsCross.riscv64.gcc14, with the intention to use the RISC-V targeting cross compiler for building an installer image for system supporting xthead extensions.

{
  nixpkgs.overlays = [
    (_final: prev: {
      gcc14 = prev.gccFromSrc {
        majorMinorVersion = "14.0.1";
        src = /path/to/gcc/repo;
      };
    })
    (_final: prev: {
      pkgsCross = lib.mapAttrs
        (_name: value: value // {
          gcc14 = value.gcc14.override {
            # NOTE: needed to build `libstdc++-v3` since it uses `std=gnu++26` which `13.2.0`
            # doesn't support, and local `xgcc` doesn't get built when `buildPlatform !=
            # hostPlatform`.
            stdenv = prev.overrideCC value.stdenv value.buildPackages.gcc14;
          };
        })
        prev.pkgsCross;
    })
  ];

  environment.systemPackages = with pkgs; [
    pkgsCross.riscv64.buildPackages.gcc14
    pkgsCross.riscv64.gcc14
  ];
}

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

@Synthetica9 @vcunat @Ericson2314 @trofi @skeuchel

[skeuchel]

It also updates some parts of the gcc definitions slightly to account for upcoming version 14. Basically the only real change necessary is that flex needs to be provided as an input.

I'm ~95% sure that flex is only needed for development versions and that the gcc 14 release tarball will come with a pre-generated lexer.

I am wondering why you do not simply override the gcc13 derivation? Since I wanted to do something like this anyway, I played with it a bit this morning and defined the following overlay:

self: super:
{
  default-gcc-version = 14;
  gcc_latest = self.gcc14;
  gcc14 = super.lowPrio (super.wrapCC (super.gcc13.cc.overrideAttrs (oldAttrs:
    let snapshot = "20240407"; in
    rec {
      name = "gcc-${version}";
      version = "14.0.0-dev.${snapshot}";
      passthru = oldAttrs.passthru // { inherit version; };
      src = super.stdenv.fetchurlBoot {
        url = "https://gcc.gnu.org/pub/gcc/snapshots/LATEST-14/gcc-14-${snapshot}.tar.xz";
        hash = "sha256-18uvcOoe1UHI2laXljf80zGBRZ9YIXN2ynSVpg7QqmU=";
      };
      nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ super.flex ];
      patches = super.lib.filter
        (patch: !super.lib.hasSuffix "ICE-PR110280.patch" (builtins.baseNameOf patch))
        oldAttrs.patches;
    })));
  gcc14Stdenv = super.overrideCC self.gccStdenv self.gcc14;
}

The filtering of patches is a bit ugly, but whatever. This allowed me to build a complete stdenv with gcc14.

$ gcc --version
gcc (GCC) 14.0.1 20240405 (experimental)
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Do you have a specific use-case in mind which is not covered by a gcc13 override?

[silvanshade]

Do you have a specific use-case in mind which is not covered by a gcc13 override?

Thanks for the feedback.

I'm still pretty new to Nix (only started about a month ago) so some things I'm still trying to understand.

I did initially try to re-use the gcc13 derivation with overrides but I couldn't get it to work. What I was missing was that it is necessary to take into account the wrapper and actually modify the underlying .cc derivation.

With that change, it seems sufficient to use overrides on gcc13 for what I was trying to achieve so I'll close this PR.

The remaining difference with the PR versus overriding gcc13 is that modifying the nativeBuildInputs does not seem to work correctly with splicing.

For example, to recreate my example above, the nativeBuildInputs for pkgsCross must also be changed (see below) because now the flex package it includes is for a riscv64-host when it should be for the native-host. Maybe there is a better way to handle that issue also?

{
  nixpkgs.overlays = [
    (final: prev:
      {
        # default-gcc-version = 14;
        gcc_latest = final.gcc14;
        gcc14 = prev.wrapCC (prev.gcc13.cc.overrideAttrs (oldAttrs:
          let snapshot = "20240407"; in
          rec {
            name = "gcc-${version}";
            version = "14.0.0-dev.${snapshot}";
            passthru = oldAttrs.passthru // { inherit version; };
            src = prev.stdenv.fetchurlBoot {
              url = "https://gcc.gnu.org/pub/gcc/snapshots/LATEST-14/gcc-14-${snapshot}.tar.xz";
              hash = "sha256-18uvcOoe1UHI2laXljf80zGBRZ9YIXN2ynSVpg7QqmU=";
            };
            nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ prev.flex ];
            patches = prev.lib.filter
              (patch: !prev.lib.hasSuffix "ICE-PR110280.patch" (builtins.baseNameOf patch))
              oldAttrs.patches;
          }));
        gcc14Stdenv = prev.overrideCC final.gccStdenv final.gcc14;
      })
    (_final: prev: {
      pkgsCross = lib.mapAttrs
        (_name: value: value // {
          gcc14 = prev.wrapCC ((value.gcc14.cc.override {
            # NOTE: needed to build `libstdc++-v3` since it uses `std=gnu++26` which `13.2.0`
            # doesn't support, and local `xgcc` doesn't get built when `buildPlatform !=
            # hostPlatform`.
            stdenv = prev.overrideCC value.stdenv value.buildPackages.gcc14;
          }).overrideAttrs (oldAttrs: {
            # NOTE: the included `flex` package is for the wrong host architecture so we need to replace it.
            nativeBuildInputs = prev.lib.filter (p: p.pname != "flex") oldAttrs.nativeBuildInputs ++ [ prev.buildPackages.flex ];
          }));
        })
        prev.pkgsCross;
    })
  ];

  environment.systemPackages = with pkgs; [
    pkgsCross.riscv64.buildPackages.gcc14
    pkgsCross.riscv64.gcc14
  ];
}

[silvanshade]

After some more experimentation I figured out how to address the last issue I mentioned, where the splicing was not working correctly.

The fix was to change the inclusion of flex to buildPackages.flex, in other words change the line in @skeuchel example:

nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ super.flex ];

to

nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ super.buildPackages.flex ];

It's also possible to do the stdenv override in the original package by using a conditional to check for the platforms.

So the full working example with cross-compilation looks like this:

{
  nixpkgs.overlays = [
    (final: prev:
      {
        gcc_latest = final.gcc14;
        gcc14 = prev.wrapCC ((prev.gcc13.cc.override (self: {
          stdenv =
            if self.stdenv.buildPlatform == self.stdenv.hostPlatform
            then self.stdenv
            # NOTE: needed to build `libstdc++-v3` since it uses `std=gnu++26` which `13.2.0`
            # doesn't support, and local `xgcc` doesn't get built when `buildPlatform !=
            # hostPlatform`.
            else prev.overrideCC self.stdenv final.buildPackages.gcc14;
        })).overrideAttrs (oldAttrs:
          let snapshot = "20240407"; in
          rec {
            name = "gcc-${version}";
            version = "14.0.0-dev.${snapshot}";
            passthru = oldAttrs.passthru // { inherit version; };
            src = prev.stdenv.fetchurlBoot {
              url = "https://gcc.gnu.org/pub/gcc/snapshots/LATEST-14/gcc-14-${snapshot}.tar.xz";
              hash = "sha256-18uvcOoe1UHI2laXljf80zGBRZ9YIXN2ynSVpg7QqmU=";
            };
            nativeBuildInputs = oldAttrs.nativeBuildInputs ++ [ prev.buildPackages.flex ];
            patches = prev.lib.filter
              (patch: !prev.lib.hasSuffix "ICE-PR110280.patch" (builtins.baseNameOf patch))
              oldAttrs.patches;
          }));
        gcc14Stdenv = prev.overrideCC final.gccStdenv final.gcc14;
      })
  ];

  environment.systemPackages = with pkgs; [
    pkgsCross.riscv64.buildPackages.gcc14
    pkgsCross.riscv64.gcc14
  ];
}