LLVMExtra JLL doesn't support USE_SYSTEM_LLVM

[CarloLucibello]

On the last release (4.1.0) I get the following error

julia> using LLVM
[ Info: Precompiling LLVM [929cbde3-209d-540e-8aea-75f648917ca0]
ERROR: LoadError: InitError: could not load library "/home/carlo/.julia/artifacts/503611619dce439df9bf3428c01561723188769b/lib/libLLVMExtra-11.so"
libLLVM-11jl.so: impossibile aprire il file oggetto condiviso: File o directory non esistente
Stacktrace:
 [1] dlopen(s::String, flags::UInt32)
   @ Base.Libc.Libdl ./libdl.jl:114
 [2] macro expansion
   @ ~/.julia/packages/JLLWrappers/bkwIo/src/products/library_generators.jl:54 [inlined]
 [3] __init__()
   @ LLVMExtra_jll ~/.julia/packages/LLVMExtra_jll/DW0cO/src/wrappers/x86_64-linux-gnu-cxx11-julia_version+1.6.0.jl:8
 [4] top-level scope (repeats 2 times)
   @ none:1
during initialization of module LLVMExtra_jll
in expression starting at /home/carlo/.julia/packages/LLVM/aqg20/src/LLVM.jl:1
ERROR: Failed to precompile LLVM [929cbde3-209d-540e-8aea-75f648917ca0] to /home/carlo/.julia/compiled/v1.6/LLVM/jl_Zx6hui.
Stacktrace:
 [1] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360

which is quite weird since libLLVMExtra-11.so is actually there

shell> ls /home/carlo/.julia/artifacts/503611619dce439df9bf3428c01561723188769b/lib/
libLLVMExtra-11.so

[jpsamaroo]

Notice the library name:

libLLVM-11jl.so: impossibile aprire il file oggetto condiviso: File o directory non esistente

The problem is that LLVMExtra can't dlopen LLVM. Can you post your versioninfo()?

[vchuravy]

How did you build Julia or where did you install Julia from?

[CarloLucibello]

julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44* (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.0 (ORCJIT, skylake)

It's the system package in my manjaro distrubution

[CarloLucibello]

Using julia 1.6.1 downloaded from the julia's website I don't experience the problem

[jpsamaroo]

That makes sense: Manjaro is likely linking Julia against the system LLVM, which doesn't match the SONAME that LLVMExtra has embedded for LLVM.

[jpsamaroo]

@vchuravy this still seems like a regression; could we somehow adjust LLVMExtra so it can just link against whatever libLLVM is already loaded?

[vchuravy]

Yes and no. we could try linking against a non-versioned variant of LLVM, but we would still assume at least a shared library built of LLVM (not necessarily the default) and a specific version of LLVM, Manjaro as an example seems to do the same as ArchLinux and link against LLVM 12 (which is not a by us supported LLVM version for Julia 1.6). So the answer is maybe, but we would first need to add a LLVM platform so that we can resolve that at runtime instead of using the Julia version as a proxy.

x-ref: SciML/SciMLSensitivity.jl#427 (comment) which is the same version, but for Enzyme.

also cc: @staticfloat and @giordano

[giordano]

Using julia 1.6.1 downloaded from the julia's website I don't experience the problem

In ArchLinux you can use aur:julia-bin

So the answer is maybe, but we would first need to add a LLVM platform so that we can resolve that at runtime instead of using the Julia version as a proxy.

That'd be similar to the CUDA tag in the platform

[staticfloat]

So the answer is maybe, but we would first need to add a LLVM platform so that we can resolve that at runtime instead of using the Julia version as a proxy.

In the long term, I would prefer to request Manjaro to set the version field of libLLM_jll to v12.x.x, and then have different releases of LLVM.jl that declare compatibility with libLLVM_jll versions. Then we can use the Pkg resolver to choose between different versions.

[vchuravy]

, and then have different releases of LLVM.jl that declare compatibility with libLLVM_jll versions.

I assume you mean LLVMExtra_jll? This for me isn't as attractive as a LLVM platform tag. The source code of LLVMExtra supports multiple different LLVM versions (same for Enzyme), it is the built artifact that has a restriction on when it can be loaded. I am assuming that you mean something like:

LLVMExtra_jll@1 => libLLVM_jll@11
LLVMExtra_jll@2 => libLLVM_jll@12
LLVMExtra_jll@3 => libLLVM_jll@13
LLVMExtra_jll@4 => libLLVM_jll@14

So what happens when I need to update the API version of LLVMExtra_jll? How do I encode this here? I really don't want to maintain 7 different version lines that map to different LLVM, I want to have one version of LLVMExtra_jll that I can load on multiple different version of LLVM.
For me the resolver is designed for API compatibility and that's what the version of LLVMExtra_jll encapsulates. The choice of artifacts is then really independent of that.

[vchuravy]

I would prefer to request Manjaro to set the version field of libLLM_jll to v12.x.x

That is also not sufficient since the symbol versioning of LLVM causes a so name mismatch

[sukanka]

I still use the julia from arch community repository. I then downloaded Julia from julia.org (or github release), unzipped it, and placed the file libLLVM-11jl.so to /home/sukanka/.julia/artifacts/f4dccfec6e4da50a7585e1174de35d00da59edc5/lib/, since it reported

ERROR: LoadError: InitError: could not load library "/home/sukanka/.julia/artifacts/f4dccfec6e4da50a7585e1174de35d00da59edc5/lib/libLLVMExtra-11.so"
libLLVM-11jl.so: xxx

Now I can using CUDA successfully, but I didn't go any further.

[maleadt]

There's a reason we don't recommend system LLVM, our version has been patched and 'verified' to emit good code, whereas vanilla LLVM has known bugs that may result in run-time problems. Since you're on arch, please use julia-bin from the AUR.

[maleadt]

I added a better error in

LLVM.jl/src/LLVM.jl

Lines 108 to 115 in 341890b

	if !isdefined(API, :libLLVMExtra)
	@error """LLVM extensions library unavailable for your platform:
	$(Base.BinaryPlatforms.triplet(API.LLVMExtra_jll.host_platform))
	LLVM.jl will not be functional.
	If you are using a custom version of LLVM, try building a
	custom version of LLVMExtra_jll using `deps/build_local.jl`"""
	end

, pointing to the scripts to build your own LLVMExtra library.