llvm lld flang wasi-runtimes 19.1.3 #196094

carlocab · 2024-10-30T13:14:16Z

Have you followed the guidelines for contributing?
Have you ensured that your commits follow the commit style guide?
Have you checked that there aren't other open pull requests for the same formula update/change?
Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

llvm 19.1.3
lld 19.1.3
flang 19.1.3
wasi-runtimes 19.1.3

Formula/l/llvm.rb

This will be used by `llvm` (and, presumably, in the future, versioned LLVM formulae). The idea is that we will write a config file for each OS version pointing to the correct SDKROOT so that `llvm` does not require rebuilding/reinstalling when a user upgrades to a new major version of macOS. See Homebrew/homebrew-core#196094.

Formula/l/llvm.rb

Bo98 · 2024-10-31T08:21:56Z

Formula/l/llvm.rb

+    arches.each do |target_arch|
+      target_triple = "#{target_arch}-apple-darwin#{kernel_version}"
+      drivers.each do |driver|
+        (clang_config_file_dir/"#{target_triple}-#{driver}.cfg").atomic_write <<~CONFIG


If you're using the clang-cl patch then you should be able to just do:

Suggested change

(clang_config_file_dir/"#{target_triple}-#{driver}.cfg").atomic_write <<~CONFIG

(clang_config_file_dir/"#{target_triple}.cfg").atomic_write <<~CONFIG

or at least that's kinda why I did the clang-cl patch.

Consider also deploying a #{arch}-apple-darwin.cfg, which will be be a fallback for unsupported versions, which uses the unversioned --sysroot=#{MacOS::CLT::PKG_PATH}/SDKs/MacOSX.sdk. Hopefully nobody needs it but I think it makes sense to do.

Right, I forgot about why we had the clang-cl patch.

Consider also deploying a #{arch}-apple-darwin.cfg, which will be be a fallback for unsupported versions, which uses the unversioned --sysroot=#{MacOS::CLT::PKG_PATH}/SDKs/MacOSX.sdk. Hopefully nobody needs it but I think it makes sense to do.

What about Xcode-only installs? We also don't handle it currently, and this change breaks them, even when building from source, but not really sure what to do about it.

It's realistically difficult/impossible to support Xcode without having a libxcselect wrapper. Apple don't even manage it without that.

$ echo | /Library/Developer/CommandLineTools/usr/bin/clang -E -Wp,-v - clang -cc1 version 16.0.0 (clang-1600.0.26.3) default target arm64-apple-darwin23.6.0 ignoring nonexistent directory "/usr/local/include" ignoring nonexistent directory "/usr/include" #include "..." search starts here: #include <...> search starts here: /Library/Developer/CommandLineTools/usr/lib/clang/16/include /Library/Developer/CommandLineTools/usr/include /System/Library/Frameworks (framework directory) /Library/Frameworks (framework directory) End of search list. $ echo | xcrun /Library/Developer/CommandLineTools/usr/bin/clang -E -Wp,-v - clang -cc1 version 16.0.0 (clang-1600.0.26.3) default target arm64-apple-darwin23.6.0 ignoring nonexistent directory "/usr/local/include" ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include" ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks" #include "..." search starts here: #include <...> search starts here: /Library/Developer/CommandLineTools/usr/lib/clang/16/include /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include /Library/Developer/CommandLineTools/usr/include /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory) End of search list.

Some people to install the latest Xcode as /Applications/Xcode-beta.app (particularly during beta cycles), some install it in ~/Applications instead, etc.

Even the existing solution here doesn't support that anyway so not sure what you mean by "breaking" it.

Well by "breaking" it, I mean that a user with an Xcode-only install at least has a chance of getting a somewhat-sensible DEFAULT_SYSROOT which comes from MacOS.sdk_path_if_needed if they build from source:

❯ sudo mv /Library/Developer/CommandLineTools . ❯ brew ruby -e 'puts MacOS.sdk_path_if_needed' /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.sdk

Now everyone for whom this worked for when they built llvm from source will still get a toolchain that refers to a non-existent path for the sysroot.

Should we just odie inside install if the CLT is not installed?

Thanks a lot for the answers! I'll open a separate issue about -syslibroot

Would also be great to also hear more on what people on Xcode-only installs think given the CLT soft requirement has been there for a while. Has brew install --build-from-source llvm been a good experience? How often do you upgrade? Is there a reason why building LLVM from source is seen as a better option to installing the CLT?

Sorry, cannot answer those questions, I'm acting here because of rust-lang/cc-rs#1278 and rust-lang/rust#131477, and hence it is somewhat important to me that Homebrew gets the details here right, but I might have been completely mistaken in certain regards (again, sorry!) because I haven't used Homebrew myself for a while.

Ah good point about missing *-apple-macosx - that's definitely worth fixing. I remembered about them initially but completely forgot when this PR was opened.

Opened #197277 and #197278

Formula/l/llvm.rb

Bo98 · 2024-10-31T08:32:09Z

Formula/l/llvm.rb

+  def post_install
+    return unless OS.mac?
+    # TODO: should we check for all config files here?
+    return if (clang_config_file_dir/"#{Hardware::CPU.arch}-apple-darwin#{OS.kernel_version.major}-clang.cfg").exist?
+
+    write_config_files(MacOS.version.major, OS.kernel_version.major, Hardware::CPU.arch)
+  end


Was initially wondering if this was really needed given we will pick up new macOS versions 3+ months prior to release but I guess makes sense for the versioned formulae that won't be version bumped.

Feels hacky to make people do brew postinstall though - brew upgrade is much more intuitive. I wonder if it makes sense to either:

Ship these files statically in brew and point the config directory to Library/Homebrew/os/mac/clang-config or something like that so that brew update will be all you need.

Ship this in a special clang-config formula or something that every LLVM formula uses.

Well the idea of postinstall was for users who didn't get a default config file for whatever reason. The idea is that it's meant to be a rare occurrence, so doing a lot more about it (e.g. files in brew, a separate formula) seems overkill.

Isn't the point to avoid people needing to avoid brew reinstall? It would indeed probably be a rare occurrence for main formula (though rare enough to probably be covered by the unversioned case), but not for versioned formulae which are rarely revision bumped.

A versioned formula carrying these changes is still several months away -- I don't think that's worth considering right now. We can handle those properly when it comes to it.

The bigger issue is Xcode-only installs, which will get broken as soon as this is merged: #196094 (comment)

Will fix this before LLVM 20, but would love more thoughts on the other thread about Xcode-only installs.

Also, backport some changes that will allow us to use config files.

Also: - avoid referencing LLVM cellar paths in symlinks - fix some build flags - add config files to simplify usage with our clang

saschasc · 2024-10-31T18:31:47Z

@carlocab I've seen that there are no bottles for Sequoia Intel (only arm64_sequoia). Would it be possible to include them as well from 19.1.3 and upcoming? This would be awesome.

carlocab · 2024-10-31T19:02:00Z

We don't have the CI for it I'm afraid, but the changes here are designed so that the :sonoma bottle works out of the box for :sequoia.

carlocab · 2024-11-01T05:27:14Z

Punting on Xcode discussions (#196094 (comment)) and non postinstall handling (#196094 (comment)).

github-actions · 2024-11-01T05:28:28Z

@carlocab has requested bottles to be published to this PR.

carlocab mentioned this pull request Oct 30, 2024

llvm: add config file, update caveats #192709

Closed

6 tasks

carlocab force-pushed the llvm-lld-flang-wasi-runtimes branch from 4e9b983 to dd13a6f Compare October 30, 2024 13:20

chenrui333 approved these changes Oct 30, 2024

View reviewed changes

cho-m reviewed Oct 30, 2024

View reviewed changes

Formula/l/llvm.rb Outdated Show resolved Hide resolved

carlocab force-pushed the llvm-lld-flang-wasi-runtimes branch 2 times, most recently from 77b8335 to 1e6a047 Compare October 30, 2024 15:35

carlocab mentioned this pull request Oct 30, 2024

macos_version: add KERNEL_MAJOR_VERSIONS map Homebrew/brew#18674

Merged

7 tasks

carlocab force-pushed the llvm-lld-flang-wasi-runtimes branch 2 times, most recently from bdf82d9 to 865984d Compare October 31, 2024 07:54

carlocab commented Oct 31, 2024

View reviewed changes

Formula/l/llvm.rb Outdated Show resolved Hide resolved

carlocab force-pushed the llvm-lld-flang-wasi-runtimes branch 2 times, most recently from ef892c7 to c1f007e Compare October 31, 2024 08:21

Bo98 reviewed Oct 31, 2024

View reviewed changes

carlocab added 4 commits October 31, 2024 16:54

llvm 19.1.3

7c77c5e

Also, backport some changes that will allow us to use config files.

lld 19.1.3

0a57c6a

flang 19.1.3

7edc8ce

wasi-runtimes 19.1.3

f347d81

Also: - avoid referencing LLVM cellar paths in symlinks - fix some build flags - add config files to simplify usage with our clang

carlocab force-pushed the llvm-lld-flang-wasi-runtimes branch from c1f007e to f347d81 Compare October 31, 2024 08:56

carlocab added long dependent tests Set a long timeout for dependent testing CI-skip-recursive-dependents Pass --skip-recursive-dependents to brew test-bot. and removed long dependent tests Set a long timeout for dependent testing labels Nov 1, 2024

carlocab added 4 commits November 1, 2024 05:32

flang: update 19.1.3 bottle.

42be020

lld: update 19.1.3 bottle.

5331b9c

llvm: update 19.1.3 bottle.

e516e42

wasi-runtimes: update 19.1.3 bottle.

471f141

github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label Nov 1, 2024

github-actions bot approved these changes Nov 1, 2024

View reviewed changes

BrewTestBot enabled auto-merge November 1, 2024 05:44

BrewTestBot added this pull request to the merge queue Nov 1, 2024

Merged via the queue into master with commit 78b4cd5 Nov 1, 2024
15 checks passed

BrewTestBot deleted the llvm-lld-flang-wasi-runtimes branch November 1, 2024 05:52

lukeshingles mentioned this pull request Nov 1, 2024

clang-tidy broken with llvm v19.1.3 #196344

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

llvm lld flang wasi-runtimes 19.1.3 #196094

llvm lld flang wasi-runtimes 19.1.3 #196094

carlocab commented Oct 30, 2024

Bo98 Oct 31, 2024

Bo98 Oct 31, 2024

carlocab Oct 31, 2024 •

edited

Loading

Bo98 Oct 31, 2024

carlocab Oct 31, 2024 •

edited

Loading

madsmtm Nov 10, 2024

Bo98 Nov 10, 2024 •

edited

Loading

madsmtm Nov 10, 2024

Bo98 Nov 10, 2024

madsmtm Nov 11, 2024

Bo98 Oct 31, 2024 •

edited

Loading

carlocab Oct 31, 2024

Bo98 Oct 31, 2024

carlocab Oct 31, 2024

carlocab Nov 1, 2024

saschasc commented Oct 31, 2024

carlocab commented Oct 31, 2024

carlocab commented Nov 1, 2024

github-actions bot commented Nov 1, 2024

	(clang_config_file_dir/"#{target_triple}-#{driver}.cfg").atomic_write <<~CONFIG
	(clang_config_file_dir/"#{target_triple}.cfg").atomic_write <<~CONFIG

llvm lld flang wasi-runtimes 19.1.3 #196094

llvm lld flang wasi-runtimes 19.1.3 #196094

Conversation

carlocab commented Oct 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carlocab Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carlocab Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Bo98 Nov 10, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Bo98 Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

saschasc commented Oct 31, 2024

carlocab commented Oct 31, 2024

carlocab commented Nov 1, 2024

github-actions bot commented Nov 1, 2024

carlocab Oct 31, 2024 •

edited

Loading

carlocab Oct 31, 2024 •

edited

Loading

Bo98 Nov 10, 2024 •

edited

Loading

Bo98 Oct 31, 2024 •

edited

Loading