Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Julia test failing (on Linux) #201

Closed
lanctot opened this issue May 4, 2020 · 7 comments
Closed

Julia test failing (on Linux) #201

lanctot opened this issue May 4, 2020 · 7 comments

Comments

@lanctot
Copy link
Collaborator

lanctot commented May 4, 2020 •
edited
Loading

Hi @findmyway , seems that as of a few days ago the julia test is failing (oddly, only the test fails... it still compiles fine). This was the original error I got

[100%] Built target spieljl
Test project /usr/local/google/home/lanctot/tmp/open_spiel/build
    Start 150: julia_test
1/1 Test #150: julia_test .......................***Failed   22.34 sec
   Testing OpenSpiel
 Resolving package versions...
/myhomedirpath/packages/julias/julia-1.3.1/bin/julia: symbol lookup error: /usr/local/google/home/lanctot/tmp/open_spiel/open_spiel/julia/deps/../../../build/julia/libspieljl.so: undefined symbol: _ZN5jlcxx19FunctionWrapperBase19set_pointer_indicesEv
ERROR: LoadError: Failed to precompile OpenSpiel [7e5d8dc8-489f-4be6-9273-f4c3010e6ec1] to /myhomedirpath/.julia/compiled/v1.3/OpenSpiel/pEQx0_MCr2V.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917
 [6] include at ./boot.jl:328 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1105
 [8] include(::Module, ::String) at ./Base.jl:31
 [9] include(::String) at ./client.jl:424
 [10] top-level scope at none:6
in expression starting at /usr/local/google/home/lanctot/tmp/open_spiel/open_spiel/julia/test/runtests.jl:1
ERROR: Package OpenSpiel errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:113
 [2] #test#131(::Bool, ::Nothing, ::Cmd, ::Cmd, ::typeof(Pkg.Operations.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1372
 [3] #test at ./none:0 [inlined]
 [4] #test#62(::Bool, ::Nothing, ::Cmd, ::Cmd, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:253
 [5] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:239
 [6] #test#58 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [7] test() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:230
 [8] top-level scope at none:1
0% tests passed, 1 tests failed out of 1
Total Test time (real) =  22.34 sec
The following tests FAILED:
	150 - julia_test (Failed)
Errors while running CTest
At least one test failed.
If this is the first time you have run these tests, try:
pip3 install -r requirements.txt
Note that outside a virtualenv, you will need to install the system 
wide matplotlib: sudo apt-get install python-matplotlib
Exiting virtualenv

Then when I removed julia and all the packages, starting from scratch (re-installing Julia via ./install.sh) I get a different error:

   Testing OpenSpiel
 Resolving package versions...
ERROR: LoadError: could not load library "/myhomedirpath/tmp/open_spiel/open_spiel/julia/deps/../../../build/julia/libspieljl"
/myhomedirpath/packages/julias/julia-1.3.1/bin/../lib/julia/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /myhomedirpathtmp/open_spiel/open_spiel/julia/deps/../../../build/julia/libspieljl.so)
Stacktrace:
 [1] #dlopen#3(::Bool, ::typeof(Libdl.dlopen), ::String, ::UInt32) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109
 [2] dlopen at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Libdl/src/Libdl.jl:109 [inlined]
 [3] readmodule(::String, ::Symbol, ::Module, ::Nothing) at /usr/local/google/home/lanctot/.julia/packages/CxxWrap/LfTHV/src/CxxWrap.jl:700
 [4] wrapmodule(::String, ::Symbol, ::Module, ::Nothing) at /usr/local/google/home/lanctot/.julia/packages/CxxWrap/LfTHV/src/CxxWrap.jl:705
 [5] top-level scope at /usr/local/google/home/lanctot/tmp/open_spiel/open_spiel/julia/src/OpenSpiel.jl:9
 [6] include at ./boot.jl:328 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1105
 [8] include(::Module, ::String) at ./Base.jl:31
 [9] top-level scope at none:2
 [10] eval at ./boot.jl:330 [inlined]
 [11] eval(::Expr) at ./client.jl:425
 [12] top-level scope at ./none:3
in expression starting at /myhomedirpath/tmp/open_spiel/open_spiel/julia/src/OpenSpiel.jl:9
ERROR: LoadError: Failed to precompile OpenSpiel [7e5d8dc8-489f-4be6-9273-f4c3010e6ec1] to /usr/local/google/home/lanctot/.julia/compiled/v1.3/OpenSpiel/pEQx0_W2oS3.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917
 [6] include at ./boot.jl:328 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1105
 [8] include(::Module, ::String) at ./Base.jl:31
 [9] include(::String) at ./client.jl:424
 [10] top-level scope at none:6
in expression starting at /myhomedirpath/tmp/open_spiel/open_spiel/julia/test/runtests.jl:1
ERROR: Package OpenSpiel errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:113
 [2] #test#131(::Bool, ::Nothing, ::Cmd, ::Cmd, ::typeof(Pkg.Operations.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1372
 [3] #test at ./none:0 [inlined]
 [4] #test#62(::Bool, ::Nothing, ::Cmd, ::Cmd, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:253
 [5] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:239
 [6] #test#58 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [7] test() at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:230
 [8] top-level scope at none:1

Did something change with the packages recently? Can you reproduce this?
@findmyway
Copy link
Collaborator

🤔 That's strange. I can't reproduce the error on my local machine or on Travis. Is there any other environment change on the build machine?

@lanctot
Copy link
Collaborator Author

lanctot commented May 4, 2020

That is certainly possible, but I'm not sure what it could be.. it has not changed much and has been up for much longer than a few days. My guess is still that either CxxWrap or something changed in the Julia packaging and the new one doesn't work with our system.

Note that Travis does not build with Julia and hence does not test it (since it's an optional dependency).

Can I confirm that this command would remove everything related to the custom Julia installation? (So I can retry from scratch)

rm -rf ~/packages/julias/ ~/.julia/ ~/.local/bin/julia*

@findmyway
Copy link
Collaborator

findmyway commented May 4, 2020 •
edited
Loading

rm -rf ~/packages/julias/ ~/.julia/ ~/.local/bin/julia*

Yes, that would be enough to remove all Julia related.

My guess is still that either CxxWrap or something changed in the Julia packaging and the new one doesn't work with our system.

I can confirm that the latest release of CxxWrap is still v0.10.1. So this might not be the real reason.

Note that Travis does not build with Julia and hence does not test it (since it's an optional dependency).

Yes, but from the log it did build with Julia with DEFAULT_OPTIONAL_DEPENDENCY="ON"

@lanctot
Copy link
Collaborator Author

lanctot commented May 7, 2020

Yes, but from the log it did build with Julia with DEFAULT_OPTIONAL_DEPENDENCY="ON"

Hmm, you're right. I take it back: it's working on the Travis machines just not internally (or might just be my machine).

This is not ideal, we don't really have a way to test it before pushing to github. I'll try again before the next update, but if you have any other ideas I'd love to hear them.

@findmyway
Copy link
Collaborator

@lanctot , I tried to search around and found a similar error with other packages here:

So is your OS also Ubuntu 19.10?

Could you try the workaround in the above link and see if it works for you?

Until this issue is fixed in Julia (see JuliaLang/julia#34276) a workaround is to replace Julias libstdc++ with the one of your OS:

# works for Ubuntu 19.10 64bit - match your locations accordingly!
cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 $JULIA_HOME/lib/julia/

I'm still using Ubuntu 18 so I can't reproduce the problem locally. I'll try in the docker environment and update what I can find here later.

@lanctot
Copy link
Collaborator Author

lanctot commented May 8, 2020

Ahh yes, I had to do that... right. And I think I got rid of that directory when we were testing other stuff, so I accidentally deleted it. Good catch! Ok, I'll try it on Monday when I go to do the update.

@lanctot
Copy link
Collaborator Author

lanctot commented May 10, 2020

Confirmed, that was indeed the problem. Thanks @findmyway!

@lanctot lanctot closed this as completed May 10, 2020
OpenSpiel pushed a commit that referenced this issue Feb 10, 2021
@jblespiau
Add CI testing using Github Actions.
af2d251 
As Travis-CI is migrating, and will remove some options for OSS projects to get free workflows, we add github actions (we can remove travis CI afterwards).

It will look like this:
https://github.com/jblespiau/open_spiel-1/actions/runs/554623616

We only have a more recent version of MacOS, with Python 3.9. The current Travis do not specify the Python 3 version, while I explicitly pin it to be Python 3.7 here.

I kept the versions the same as in .travis.yml otherwise (we have a Python 3.6, for Nox testing, we should likely upgrade it to be 3.7, but I would prefer this to be an orthogonal change.

I just have a failure for the test with Julia:
https://github.com/jblespiau/open_spiel-1/runs/1595264912?check_suite_focus=true

It should be easy to fix this using in the install script:

```
  # This is needed on Ubuntu 19.10 and above, see:
  # #201
  if [[ "$JULIA_HOME" != "$HOME/packages/julias/julia-1.3.1" ]]; then
    if [[ -f /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ]]; then
      echo "Julia not in $HOME. Copy libstdc++.so.6 (requires sudo)"
      sudo cp -f /usr/lib/x86_64-linux-gnu/libstdc++.so.6 $JULIA_HOME/lib/julia
```

but I simply disabled the test in this PR, and we can follow-up with that.

PiperOrigin-RevId: 356724390
Change-Id: I69141b172c73333daaefd60534376e38dc36fdea
@lanctot lanctot mentioned this issue Feb 11, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lanctot @findmyway