Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track UT gaps for LLVM-path vs SPIRV-path. #223

Closed
tdeng5 opened this issue Jan 10, 2024 · 4 comments
Closed

Track UT gaps for LLVM-path vs SPIRV-path. #223

tdeng5 opened this issue Jan 10, 2024 · 4 comments
Assignees

Comments

@tdeng5
Copy link

tdeng5 commented Jan 10, 2024

Issue Description

Collect the latest UT report for both LLVM-path and SPIRV-path, check the failed cases. High priority for fixing those bugs which failed in LLVM-path while pass in SPIRV-path.

Code Example

System Info

  • OS:
  • PyTorch:
  • Intel Extension For Pytorch:
  • LLVM Version.
  • Triton Version:
  • Intel® XPU Backend for Triton*:
@ESI-SYD
Copy link
Contributor

ESI-SYD commented Jan 10, 2024

UT Status in main branch SPIRV-path

============ Overview ============
num_language: 6859
num_others: 901
num_total: 7760
num_skipped: 1158
num_cuda_only_skipped: 637
num_failed: 995
num_passed: 5607
total_pass_rate: 77.49%
num_new_solved_expected_cases: 5
num_new_solved_todo_cases: 32
num_new_failed_cases: 943
num_new_failed_not_todo_cases: 0

failed cases

UT failed cases in llvm-target branch: llvm-path

============ Overview (core tests) ============
=== 5422 passed, 392 skipped, 1165 xfailed, 56 warnings in 816.25s (0:13:36) ===
=== 5422/(5422+392+1165) = 5422/6979 = 77.69% ===

xfail cases

test_core.py::test_atomic_rmw #152
test_core.py::test_convert2d [skip]
test_core.py::test_convertmma2mma [test_mma2mma is not supported in HIP/XPU]
test_core.py::test_dot [skip]
test_core.py::test_fp8_dot_acc [test_fp8_dot_acc not supported on XPU]
test_core.py::test_reduce_layouts #156

cuda only

test_core.py::test_globaltimer
test_core.py::test_num_threads
test_core.py::test_smid
test_core.py::test_inline_asm[1]
test_core.py::test_inline_asm_multiple_outputs
test_core.py::test_inline_asm_packed[1]
test_core.py::test_inline_asm_packed_multiple_outputs
test_core.py::test_inline_asm_with_pointers[1]

skipped as FIXME issue

test_core.py::test_scan2d #153
test_core.py::test_locality #154
test_core.py::test_scan_layouts #155
test_core.py::test_reduce_layouts #156
test_core.py::test_chain_reduce #164
test_core.py::test_masked_load_shared_memory #158
test_conversions.py::test_typeconvert_downcast #212
test_conversions.py::test_typeconvert_upcast #213

Note

No more cases passed in spirv-path but failed in llvm-path in language cases.

Non-language cases not ported and enabled in llvm-path

operator: #167 runtime: #206 tools: cuda releated

Non-language cases ported and PASSED in SPIRV-PATH (issues tracked)

operators/test_blocksparse.py::test_softmax[64-1871-False]
operators/test_blocksparse.py::test_softmax[128-2511-False]
operators/test_blocksparse.py::test_softmax[32-576-False]
operators/test_blocksparse.py::test_softmax[16-256-False]

operators/test_blocksparse.py::test_attention_fwd_bwd[dtype0-16]
operators/test_blocksparse.py::test_attention_fwd_bwd[dtype1-16]
operators/test_blocksparse.py::test_attention_fwd_bwd[dtype0-32]
operators/test_blocksparse.py::test_attention_fwd_bwd[dtype1-32]
operators/test_blocksparse.py::test_attention_fwd_bwd[dtype0-64]
operators/test_blocksparse.py::test_attention_fwd_bwd[dtype1-64]

@etiotto
Copy link
Contributor

etiotto commented Jan 15, 2024

Overall I would say no surprises here. This information was known already, I have opened issues to track the functional problems which are also linked in this work item.

@ESI-SYD
Copy link
Contributor

ESI-SYD commented Jan 16, 2024

closing, find specified work item from opening issues, feel free to reopen

@ESI-SYD ESI-SYD closed this as completed Jan 16, 2024
@tdeng5 tdeng5 changed the title Collect the latest UT report for both LLVM-path and SPIRV-path, analysis the failed cases. Track UT gaps for both LLVM-path and SPIRV-path. Jan 19, 2024
@ESI-SYD ESI-SYD reopened this Jan 19, 2024
@tdeng5 tdeng5 changed the title Track UT gaps for both LLVM-path and SPIRV-path. Track UT gaps for LLVM-path vs SPIRV-path. Jan 30, 2024
@vlad-penkin
Copy link
Contributor

llvm branch have reached the parity with the spirv on the ut pass rate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants