Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: install luacov & cluacov to spec tree #1984

Conversation

benoit-pierre
Copy link
Contributor

@benoit-pierre benoit-pierre commented Nov 21, 2024

This fix coverage on master (https://app.circleci.com/pipelines/github/koreader/koreader/11659/workflows/527da8a7-fe87-4125-8656-dd1a707a2a9e/jobs/17231). The addition of cluacov (which must be compiled for the luajit version used during coverage, or things go boom) also greatly speeds up things (see https://app.circleci.com/pipelines/github/benoit-pierre/koreader/528/workflows/59965b76-8cfa-4e07-a75e-454aefc6c088/jobs/547).


This change is Reviewable

@Frenzie Frenzie merged commit fccfa33 into koreader:master Nov 22, 2024
4 checks passed
@benoit-pierre benoit-pierre deleted the pr/install_luacov_and_cluacov_to_spec_tree branch November 22, 2024 15:41
benoit-pierre added a commit to benoit-pierre/koreader that referenced this pull request Nov 22, 2024
@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

Random PR picked maybe (although it mentions luacov which is in my output below too), just to mention that when attempting to re kodev build, some luarock stuff (I think) do get rebuilt each time:

make TARGET= KODEBUG=1 VERBOSE=
   0% | Building 'crengine'
ninja: no work to do.
   4% | Building 'busted'
No existing manifest. Attempting to rebuild...
busted 2.2.0-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/busted/build/dist (license: MIT <http://opensource.org/licenses/MIT>)

   8% | Installing 'crengine'
  12% | Building 'cluacov'
No existing manifest. Attempting to rebuild...
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/cluacov/deepactivelines.c -o src/cluacov/deepactivelines.o
gcc  -shared -o /tmp/luarocks_build-cluacov-0.1.4-1-7688475/cluacov/deepactivelines.so src/cluacov/deepactivelines.o
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/cluacov/hook.c -o src/cluacov/hook.o
gcc  -shared -o /tmp/luarocks_build-cluacov-0.1.4-1-7688475/cluacov/hook.so src/cluacov/hook.o
cluacov 0.1.4-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/cluacov/build/dist (license: MIT)

  16% | Installing 'busted'
  20% | Building 'dkjson'
No existing manifest. Attempting to rebuild...
dkjson 2.8-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/dkjson/build/dist (license: MIT/X11)

  25% | Installing 'cluacov'
  29% | Building 'lua_cliargs'
No existing manifest. Attempting to rebuild...
lua_cliargs 3.0-2 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/lua_cliargs/build/dist (license: MIT <http://opensource.org/licenses/MIT>)

  33% | Installing 'dkjson'
  37% | Building 'luacov'
No existing manifest. Attempting to rebuild...
luacov 0.15.0-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/luacov/build/dist (license: MIT)

  41% | Installing 'lua_cliargs'
  45% | Building 'luafilesystem'
No existing manifest. Attempting to rebuild...
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/lfs.c -o src/lfs.o
gcc  -shared -o /tmp/luarocks_build-LuaFileSystem-1.8.0-1-4451826/lfs.so src/lfs.o
luafilesystem 1.8.0-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/luafilesystem/build/dist (license: MIT/X11)

  50% | Installing 'luacov'
  54% | Building 'luassert'
No existing manifest. Attempting to rebuild...
luassert 1.9.0-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/luassert/build/dist (license: MIT <http://opensource.org/licenses/MIT>)

  58% | Installing 'luafilesystem'
  62% | Building 'luasystem'
No existing manifest. Attempting to rebuild...
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/core.c -o src/core.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/compat.c -o src/compat.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/time.c -o src/time.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/environment.c -o src/environment.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/random.c -o src/random.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/term.c -o src/term.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/bitflags.c -o src/bitflags.o -I/usr/include
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=base/= -march=native -fPIC -Ibase/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/wcwidth.c -o src/wcwidth.o -I/usr/include
gcc  -shared -o /tmp/luarocks_build-luasystem-0.4.4-1-5657286/system/core.so src/core.o src/compat.o src/time.o src/environment.o src/random.o src/term.o src/bitflags.o src/wcwidth.o -L/usr/lib/x86_64-linux-gnu -Wl,-rpath,/usr/lib/x86_64-linux-gnu -lrt
luasystem 0.4.4-1 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/luasystem/build/dist (license: MIT <http://opensource.org/licenses/MIT>)

  66% | Installing 'luassert'
  70% | Building 'penlight'
No existing manifest. Attempting to rebuild...
penlight 1.14.0-2 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/penlight/build/dist (license: MIT/X11)

  75% | Installing 'luasystem'
  79% | Building 'say'
No existing manifest. Attempting to rebuild...
say 1.4.1-3 is now installed in base/build/x86_64-linux-gnu-debug/thirdparty/say/build/dist (license: MIT)

  83% | Installing 'penlight'
  87% | Building 'koreader'
ninja: no work to do.
  95% | Installing 'koreader'
 100% | Installing 'say'
[*] install front spec only for the emulator
[*] Install update once marker
[*] Install plugins
[*] Install resources
[*] Install data files

(I also got tesseract rebuilt, while I remember it did get built last time I synbed 3 days ago.)
Do I need to execute something for this to not happen?

@benoit-pierre
Copy link
Contributor Author

Well, the tesseract update was bumped to master only 32 hours ago, same with the luarocks stuff.

You get the same output every time?

@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

You get the same output every time?

What I pasted above: yes.
(But not tesseract, ok, I may have not see there was a bump and you were active :))

@benoit-pierre
Copy link
Contributor Author

You can try running make TARGET= KODEBUG=1 VERBOSE= NINJAFLAGS='-d explain' to get some debug information from ninja.

Frenzie pushed a commit to koreader/koreader that referenced this pull request Nov 22, 2024
@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

It doesnt really explain anything to me :)
Some extract of the huge output:

ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/turbo-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/turbo-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/turbo-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/turbo-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/turbo is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/turbo/all is dirty
ninja explain: depfile '/space/kobo/koreader/base/build/x86_64-linux-gnu-debug/cmake/CMakeFiles/d/b2c7cf47a0216d70faba8543243972a366aa524c0841f989e472150f2ca6c1b0.d' is missing
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luajit/luajit is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/CMakeFiles/busted-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/CMakeFiles/busted-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/busted is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/all is dirty
ninja explain: depfile '/space/kobo/koreader/base/build/x86_64-linux-gnu-debug/cmake/CMakeFiles/d/0cfd4e4abaadd8393abfaa0cf815b6c4580984d02c0dee8b7280ff403dd72c74.d' is missing
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/cluacov/cluacov-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/cluacov/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/cluacov/CMakeFiles/cluacov-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/cluacov/stamp/build is dirty
[...]
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/penlight/penlight-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/penlight/penlight is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/penlight/all is dirty
ninja explain: depfile '/space/kobo/koreader/base/build/x86_64-linux-gnu-debug/cmake/CMakeFiles/d/a6c2897ac20c816fbf5af148b5bd22a028da9734a38c333610d8d1fda28a485d.d' is missing
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/luarocks/luarocks is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/CMakeFiles/say-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/CMakeFiles/say-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say-install is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/say is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/say/all is dirty
ninja explain: output koreader/koreader-deps of phony edge with no inputs doesn't exist
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/libjpeg-turbo/libjpeg-turbo is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-configure is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-deps is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre-install is dirty
ninja explain: koreader/koreader-deps is dirty
ninja explain: output koreader/CMakeFiles/koreader-build doesn't exist
ninja explain: koreader/CMakeFiles/koreader-build is dirty
ninja explain: koreader/stamp/build is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/crengine/crengine is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/djvulibre/djvulibre is dirty
ninja explain: /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/freetype2/freetype2 is dirty
[...]
ninja explain: koreader/koreader-configure is dirty
ninja explain: koreader/koreader-build is dirty
ninja explain: koreader/koreader-configure is dirty
ninja explain: koreader/koreader-deps is dirty
ninja explain: koreader/stamp/build is dirty
ninja explain: koreader/CMakeFiles/koreader-install is dirty
ninja explain: koreader/stamp/install is dirty
ninja explain: koreader/koreader-build is dirty
ninja explain: koreader/koreader-install is dirty
ninja explain: koreader/koreader is dirty
ninja explain: koreader/all is dirty
⸠ 0% | Building 'crengine'
ninja: no work to do.
⸠ 4% | Building 'busted'
No existing manifest. Attempting to rebuild...
busted 2.2.0-1 is now installed in /space/kobo/koreader/base/build/x86_64-linux-gnu-debug/thirdparty/busted/build/dist (license: MIT <http://opensource.org/licenses/MIT>)

⸠12% | Building 'cluacov'
No existing manifest. Attempting to rebuild...
gcc -Og -g -pipe -fno-omit-frame-pointer -ffile-prefix-map=/space/kobo/koreader/base/= -march=native -fPIC -I/space/kobo/koreader/base/build/x86_64-linux-gnu-debug/staging/include/luajit-2.1 -c src/cluacov/hook.c -o src/cluacov/hook.o

@benoit-pierre
Copy link
Contributor Author

Can you attach the full output?

@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

Sure: explain.txt

@benoit-pierre
Copy link
Contributor Author

And remind me your cmake version?

@benoit-pierre
Copy link
Contributor Author

And ninja version.

@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

$ cmake --version
cmake version 3.25.1
CMake suite maintained and supported by Kitware (kitware.com/cmake).

$ ninja --version
1.11.1

@benoit-pierre
Copy link
Contributor Author

benoit-pierre commented Nov 22, 2024

I can reproduce with cmake 3.25.1.

@benoit-pierre
Copy link
Contributor Author

Can you confirm #1986 fix your issue?

You'll have to rebuild those external projects from scratch after applying the fix: make TARGET= KODEBUG=1 VERBOSE= busted-re cluacov-re dkjson-re lua-term-re lua_cliargs-re luacov-re luafilesystem-re luassert-re luasystem-re mediator_lua-re penlight-re say-re. Subsequent calls should not rebuild them, e.g. make TARGET= KODEBUG=1 VERBOSE= busted should have ninja reporting there's nothing to do.

@poire-z
Copy link
Contributor

poire-z commented Nov 22, 2024

^ Following your procedure, it's all fine - after it:

make TARGET= KODEBUG=1 VERBOSE=
⸠ 0% | Building 'crengine'
ninja: no work to do.
⸠50% | Building 'koreader'
ninja: no work to do.
â¸100% | Installing 'koreader'
[*] install front spec only for the emulator
[*] Install update once marker
[*] Install plugins
[*] Install resources
[*] Install data files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants