[CI] Fix missing deps in dev igc installation #13610
Conversation
All opencl tests with current dev igc installtation are failing. Investigation shows that we are missing libopencl-clang. Unfortunately dev igc deb package did not include libopencl-clang, and Ubuntu did not have the correct version either, apt has up to libopencl-clang13, while we need libopenc-clang14. So the workaround is to backup the version installed by released igc version, then install it back after installing dev igc. This also seperate the installation of dev igc to new step, so that we can always do checksum and remove force dependency option to dpkg.
|
Joint matrix tests passed https://github.com/intel/llvm/actions/runs/8904683281/job/24454373828?pr=13610. |
devops/scripts/install_drivers.sh
Outdated
| # Dev IGC deb package did not include libopencl-clang | ||
| # So we need to backup and install it from release igc. |
There was a problem hiding this comment.
@bashbaug , do you know who can fix that on the IGC side?
There was a problem hiding this comment.
I would suggest starting with @pszymich.
There was a problem hiding this comment.
Thanks @aelovikov-intel and @bashbaug ! I had a quick look at https://github.com/intel/intel-graphics-compiler/blob/b077b43fa6193c97bcd2f3959f6d11e185d6fef3/.github/workflows/build-IGC.yml#L39C1-L41C56 , looks like IGC is also doing the hacky install/copy/package due to the fact that there is no libopencl-clang deb available.
I have contacted the libopencl-clang team to see whether we can build/release packaged libopencl-clang deb file in https://github.com/intel/opencl-clang/releases too. If that is fixed, then we should be able to remove this workaround, and also the workaround in IGC CI.
| @@ -60,7 +60,9 @@ jobs: | |||
| - name: Determine Arc tests | |||
| id: arc_tests | |||
| run: | | |||
| if [ "${{ contains(needs.detect_changes.outputs.filters, 'drivers') }}" == "true" ]; then | |||
| if [ "${{ contains(needs.detect_changes.outputs.filters, 'devigccfg') }}" == "true" ]; then | |||
| echo 'arc_tests="Matrix/"' >> "$GITHUB_OUTPUT" | |||
There was a problem hiding this comment.
I'd think we need to run all tests here as well.
There was a problem hiding this comment.
There are existing failures for non joint matrix tests.
eg: https://github.com/intel/llvm/actions/runs/8903576097/job/24452097484
We will only use igc dev driver for joint matrix for now, so we don't really need to run all e2e test,
only make sure joint matrix tests are clean is enough.
Running all tests would be an ideal solution, but to be honest, I don't think we have enough resource to triage and response to failures, especially in non joint matrix ones, so running all tests might effectively stop us from updating igc dev driver for joint matrix tests.
There was a problem hiding this comment.
What about ESIMD team's tests? + @sarnex
There was a problem hiding this comment.
TBH our tests take a long time to run. What is the use case of using the dev IGC driver? If I know that I can suggest if we need to run ESIMD tests with it.
There was a problem hiding this comment.
The dev IGC driver will be used in joint matrix pre-commit CI tests so that we can test some features that depends on newer IGC and get integrated test feedback earlier than before.
I would personally recommend that we also enable the pre-commit CI for ESIMD, because ESIMD features tend to depends on corresponding IGC changes as well. But it is up to your team to decide.
The cost of enabling ESIMD test here is that we may need to response to potential failures in CI , it would be great if the failures are real regressions that we need to resolve, but there will definitely be noise due to IGC changes etc.
There was a problem hiding this comment.
Thanks and yeah I agree ESIMD is very sensitive to IGC versions as well. I think we should enable dev IGC driver testing in ESIMD.
@v-klochkov FYI
|
@sarnex Would you please have a look at the workaround in sycl/test-e2e/ESIMD/hardware_dispatch.cpp . I had a detail look, it should be due to some tricky CI env exposed some unknown spirv tools bug -- I can reproduce it using spv, I built the igc using exact same steps and buildIGC.sh script within same docker as used in igc repo, it is passing. I also build it manually, it is also passing. Given it is passing in personal build, I think it is hard to get it fixed in CI. So I would propose we add the workaround to by pass the auto detection logic -- that is not our test point either. This can help us set up a baseline for the pre-commit CI for dev igc. Thanks. |
|
@jsji Workaround seems fine to me. |
|
@intel/llvm-gatekeepers Can someone help merge this? Thanks. |
All opencl tests with current dev igc installtation are failing.
Investigation shows that we are missing libopencl-clang.
Unfortunately dev igc deb package did not include libopencl-clang,
and Ubuntu did not have the correct version either, apt has up to
libopencl-clang13, while we need libopenc-clang14.
So the workaround is to backup the version installed by released igc
version, then install it back after installing dev igc.
This also seperate the installation of dev igc to new step,
so that we can always do checksum and remove force dependency option to dpkg.