Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add prefill 8B f16 torch sdpa test, update tests with compile flags and tp flags, with nightly iree #456

Merged
merged 12 commits into from
Nov 20, 2024
Merged

Add prefill 8B f16 torch sdpa test, update tests with compile flags and tp flags, with nightly iree #456

merged 12 commits into from
Nov 20, 2024

Conversation

aviator19941
Copy link
Collaborator

@aviator19941 aviator19941 commented Nov 8, 2024 •
edited
Loading

  1. Update tests with compile flags and tp flags, with nightly iree.
  2. Only use 8b f16 decomposed test to decrease the CI wait time.

ScottTodd
ScottTodd previously requested changes Nov 8, 2024
View reviewed changes
.github/workflows/ci-llama-large-tests.yaml Outdated Show resolved Hide resolved
sharktank/sharktank/utils/export_artifacts.py Outdated Show resolved Hide resolved
@aviator19941 aviator19941 force-pushed the update-ci-and-llama-flags branch from 3c8ee76 to e1ae67a Compare November 8, 2024 21:43
@aviator19941 aviator19941 force-pushed the update-ci-and-llama-flags branch 2 times, most recently from 9722977 to 2cd8640 Compare November 14, 2024 01:59
@aviator19941 aviator19941 changed the title Update tests with compile flags and tp flags, with nightly iree Add prefill torch sdpa test, update tests with compile flags and tp flags, with nightly iree Nov 14, 2024
@aviator19941 aviator19941 changed the title Add prefill torch sdpa test, update tests with compile flags and tp flags, with nightly iree Add prefill 8B f16 torch sdpa test, update tests with compile flags and tp flags, with nightly iree Nov 14, 2024
@ScottTodd ScottTodd dismissed their stale review November 14, 2024 02:57

unblock request changes

@aviator19941 aviator19941 force-pushed the update-ci-and-llama-flags branch 2 times, most recently from 1af1efc to 06ad717 Compare November 19, 2024 16:50
@aviator19941
Update tests with compile flags and tp flags, try nightly iree
7a5ee68 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Fix formatting, remove pdb
b13f855 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update CI comments and args in tests
2ce4d63 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Add non-decomposed 8b f16 prefill only test nightly
f47401d 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update preprocess pass compile flag
c3522aa 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update iree nightly, add prefill torch sdpa test
7695df1 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Get nightly iree, keep iree-turbine numpy
c6df3df 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update iree packages to nightly, fix name of test, create README
d8872e2 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Fix pytest option dest and missed merge conflict
933d4a6 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Move --skip-decode to model cli helper
09c5220 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update tests, TODO: decomposed failing with OUT OF RANGE command buff…
ecf4026 
…er validation

Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941
Update flags and github pages publish_dir and destination_dir
350ff47 
Signed-off-by: aviator19941 <avinash.sharma@amd.com>
@aviator19941 aviator19941 force-pushed the update-ci-and-llama-flags branch from a169585 to 350ff47 Compare November 20, 2024 01:39
@aviator19941 aviator19941 merged commit 4147e5e into main Nov 20, 2024
5 checks passed
@aviator19941 aviator19941 deleted the update-ci-and-llama-flags branch November 20, 2024 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marbre marbre marbre left review comments

@ScottTodd ScottTodd ScottTodd left review comments

@saienduri saienduri saienduri approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@aviator19941 @ScottTodd @marbre @saienduri