Fix GHC loadArchive clang++ error on macOS #149942

jkachmar · 2021-12-09T23:41:16Z

Motivation for this change

Things done

Built on platform(s)
- x86_64-linux
- aarch64-linux
- x86_64-darwin
- aarch64-darwin
For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
Tested, as applicable:
- NixOS test(s) (look inside nixos/tests)
- and/or package tests
- or, for functions and "core" functionality, tests in lib/tests or pkgs/test
- made sure NixOS tests are linked to the relevant packages
Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)
- (Package updates) Added a release notes entry if the change is major or breaking
- (Module updates) Added a release notes entry if the change is significant
- (Module addition) Added a release notes entry if adding a new NixOS module
- (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
Fits CONTRIBUTING.md.

jkachmar · 2021-12-09T23:42:01Z

Note that I haven't tested this patch on GHC 8.8.4, 9.0.1, or 9.2.1.

I've only applied it to those expressions following IOHK's guidance (as they apply this patch in their repository to all versions of GHC following 8.6.4).

EDIT: If possible I would really appreciate it if we could back port this to earlier versions of nixpkgs (21.11 at a minimum, but 21.05 would also be wonderful).

I was bitten by it on x86_64-darwin before (but I wasn't sufficiently motivated to track down a solution at the time); today I ran into it again on aarch64-darwin now that I'm trying to use nixpkgs with one of the new M1 MacBook Pros.

GHC 8.6.4 introduced an issue where linking to `-libc++` will cause GHC to resolve the library patch to `c++`, which in turn is a link to clang++. This results in the following error: ``` ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/?-clang-wrapper-5.0.2/bin/clang++' <no location info>: error: loadArchive "/nix/store/?-clang-wrapper-5.0.2/bin/clang++": failed ``` This commit applies the relevant patch to revert GHC's linking behavior on darwin for the applicable versions of GHC (i.e. after 8.6.4).

jkachmar · 2021-12-10T18:19:30Z

@bgamari made the following observation when we discussed this in another channel:

Oof, that c++ issue is rather frustrating since it's arguably a workaround for a linker bug which may break currently-working programs

If I understand correctly this patch will break users who specify, e.g., extra-libraries: z in their cabal file

I think a more correct way to work around this bug would be to incorporate a check to ensure that we aren't trying to link against an executable

cdepillabout · 2021-12-14T00:33:30Z

If I understand correctly this patch will break users who specify, e.g., extra-libraries: z in their cabal file

@jkachmar Were you able to figure out if this is the case?

If so, it sounds pretty bad? My guess is that there are quite a few packages using extra-libraries (although I'm not actually that sure).

Also, most of the Haskell maintainers (other than sternenseemann) don't use darwin as far as I know, so we might have to ping @ NixOS/darwin-maintainers in order to get this reviewed.

jkachmar · 2022-02-09T04:04:30Z

I looked into this a bit more again tonight and I still don't really know what the best path forwards is here.

Some related issues:

@jwiegley I saw that you opened the GHC issue linked above; did you ever find a satisfactory workaround to the issue (or was it resolved for you in an updated GHC release)?

mariatsji · 2022-06-15T11:56:03Z

Thanks, this patch solved the problem I was having with cabal build in a nix-shell using GHC 8.10.7 on darwin:

ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/cvz17hkb0wpbsyx58l4zcaamqs2bhlmz-clang-wrapper-11.1.0/bin/clang++'
ghc: panic! (the 'impossible' happened)
  (GHC version 8.10.7:
        loadArchive "/nix/store/cvz17hkb0wpbsyx58l4zcaamqs2bhlmz-clang-wrapper-11.1.0/bin/clang++": failed

Improvements to the Nix configuration so that macOS is supported. Microsoft SQL Server is still not supported (yet; I have something in mind there), but the rest works. You can still use Homebrew to install the SQL Server drivers. I had to make the following changes: * I updated nixpkgs, because it's been a while. * I made `ODBCINSTINI` optional so that it's not loaded on macOS, as it depends on `msodbcsql17`, which is broken on macOS. * I upgraded OpenSSL. * I set `DYLD_LIBRARY_PATH` on macOS so GHC finds OpenSSL; otherwise, it uses the wrong version of `libcrypto`, and fails with a fun error: > WARNING: ghc is loading libcrypto in an unsafe way * I patched GHC to fix compilation on macOS (copied from NixOS/nixpkgs#149942). To test this out, you can run `nix develop` (or install [direnv][] and add `use flake` to _.envrc.local_), and then try building HGE in the shell provided. Fair warning: GHC needs to be built, and takes _aaages_ the first time. If this becomes useful to others, we can set up a shared cache. [direnv]: https://direnv.net/ PR-URL: hasura/graphql-engine-mono#5989 GitOrigin-RevId: 7130738d47709c37778b31c134061758ce23e959

ghost · 2022-12-01T18:14:51Z

A bit surprised this is still not fixed. Can't build cabal-install on darwin because of that. :(

sternenseemann · 2022-12-01T19:15:39Z

As stated above, the workaround is not satisfactory, as it may break legitimate use cases that are already working today.

Building cabal-install on darwin is unaffected by this issue:

jkachmar · 2022-12-02T00:46:24Z

i agree that this solution is unsatisfactory, however as far as i can tell "something" does need to be done to address this. i'm fine with closing this PR, but i hesitated to do so because it has more activity (and more context) than the discussion at #149937.

if anyone has any ideas for how we should fix this, i'd be more than willing to dedicate some time to exploring them and patching things properly.

There's a transitive dependency somewhere that links against c++ which causes all kinds of havoc. For now, easy way out: use NixOS/nixpkgs#149942

maksar · 2023-03-18T15:27:18Z

iohk repo doesn't have a patch for ghc944. Structure of the files has changed in GHC. I used this patch and can confirm issue disappeared:

--- a/compiler/GHC/Linker/Loader.hs
+++ b/compiler/GHC/Linker/Loader.hs
@@ -1616,8 +1616,6 @@
        | verbatim = [lib]
        | otherwise = [ "lib" ++ lib ++ lib_tag <.> "a"
                      , lib <.> "a" -- native code has no lib_tag
-                     , "lib" ++ lib
-                     , lib
                      ]
      lib_tag = if is_hs && loading_profiled_hs_libs then "_p" else ""

sternenseemann · 2023-03-18T17:39:03Z

@maksar This is equivalent to the patch proposed in this PR and as mentioned above, GHC upstream thinks that the patch breaks legitimate use cases.

harryaskham · 2024-06-14T22:09:59Z

Experiencing the same issue with GHC 9.8.1 on nix-darwin, tried with gcc/clang from each of OSX and nixpkgs in the path. I'll see if patching still works following e.g. e9d090f

maksar · 2024-06-15T12:53:33Z

@harryaskham JFYI we are still using this patch on 9.8.2 and it helps on darwin and doesn't hurt on linux.

jkachmar requested review from cdepillabout, expipiplus1, maralorn and sternenseemann as code owners December 9, 2021 23:41

github-actions bot added the 6.topic: haskell label Dec 9, 2021

ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Dec 10, 2021

ofborg bot requested a review from guibou December 10, 2021 18:18

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 0 labels Dec 10, 2021

cdepillabout mentioned this pull request Dec 31, 2021

haskellPackages: update stackage and hackage #152408

Merged

sternenseemann marked this pull request as draft December 1, 2022 19:11

zwilias added a commit to NoRedInk/servant-elm that referenced this pull request Feb 3, 2023

Use nixpkgs fork with ghc patch

00e8c41

There's a transitive dependency somewhere that links against c++ which causes all kinds of havoc. For now, easy way out: use NixOS/nixpkgs#149942

jkachmar closed this by deleting the head repository Nov 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix GHC loadArchive clang++ error on macOS #149942

Fix GHC loadArchive clang++ error on macOS #149942

jkachmar commented Dec 9, 2021 •

edited

Loading

jkachmar commented Dec 9, 2021 •

edited

Loading

jkachmar commented Dec 10, 2021

cdepillabout commented Dec 14, 2021

jkachmar commented Feb 9, 2022

mariatsji commented Jun 15, 2022

ghost commented Dec 1, 2022 •

edited by ghost

Loading

sternenseemann commented Dec 1, 2022

jkachmar commented Dec 2, 2022

maksar commented Mar 18, 2023

sternenseemann commented Mar 18, 2023

harryaskham commented Jun 14, 2024

maksar commented Jun 15, 2024

Fix GHC loadArchive clang++ error on macOS #149942

Fix GHC loadArchive clang++ error on macOS #149942

Conversation

jkachmar commented Dec 9, 2021 • edited Loading

Motivation for this change

Things done

jkachmar commented Dec 9, 2021 • edited Loading

jkachmar commented Dec 10, 2021

cdepillabout commented Dec 14, 2021

jkachmar commented Feb 9, 2022

mariatsji commented Jun 15, 2022

ghost commented Dec 1, 2022 • edited by ghost Loading

sternenseemann commented Dec 1, 2022

jkachmar commented Dec 2, 2022

maksar commented Mar 18, 2023

sternenseemann commented Mar 18, 2023

harryaskham commented Jun 14, 2024

maksar commented Jun 15, 2024

jkachmar commented Dec 9, 2021 •

edited

Loading

jkachmar commented Dec 9, 2021 •

edited

Loading

ghost commented Dec 1, 2022 •

edited by ghost

Loading