Skip to content

Commit

Permalink
[CI] Create Github Job Summary for failing E2E tasks (#13069)
Browse files Browse the repository at this point in the history
Unfortunately, our E2E tests use custom LIT format so I wasn't able to
make `llvm-lit` generate the results into a JSON file. That meant that I
have to redirect the entire output into a log file and Github Actions
won't be able to show intermediate progress until `llvm-lit` finishes
execution.

Another thing worth mentioning is that these summaries aren't
"searchable" via Github's search queries (unlike comments), but I think
it's still a usability improvement because it would allow to see all the
failures on a single page.
  • Loading branch information
aelovikov-intel committed Mar 20, 2024
1 parent 5cfbc73 commit a4334da
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions .github/workflows/sycl-linux-run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -268,12 +268,19 @@ jobs:
run: |
cmake -GNinja -B./build-e2e -S./llvm/sycl/test-e2e -DSYCL_TEST_E2E_TARGETS="${{ inputs.target_devices }}" -DCMAKE_CXX_COMPILER="$(which clang++)" -DLLVM_LIT="$PWD/llvm/llvm/utils/lit/lit.py" ${{ steps.cmake_opts.outputs.opts }}
- name: SYCL End-to-end tests
shell: bash {0}
if: inputs.tests_selector == 'e2e'
env:
LIT_OPTS: -v --no-progress-bar --show-unsupported --show-pass --show-xfail --max-time 3600 --time-tests ${{ inputs.extra_lit_opts }}
run: |
ninja -C build-e2e check-sycl-e2e
ninja -C build-e2e check-sycl-e2e > e2e.log 2>&1
exit_code=$?
cat e2e.log
if [ $exit_code -ne 0 ]; then
# TODO: XPASS/UNRESOLVED
awk '/^Failed Tests|Testing Time/{flag=1}/FAILED: CMakeFiles/{flag=0}flag' e2e.log >> $GITHUB_STEP_SUMMARY
fi
exit $exit_code
- name: Build SYCL CTS tests
if: inputs.tests_selector == 'cts'
env:
Expand Down

0 comments on commit a4334da

Please sign in to comment.