Enable AMD BF16 Grouped Gemm #3526
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for pytorch-fbgemm-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
jwfromm
force-pushed
the
export-D67261862
branch
from
6ae3c7a to
26ab8e8
Compare
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
Summary: This diff cleans up some of the APIs for FBGEMM grouped gemm and updates CUTLASS bf16 grouped gemm to use a single kernel launch to initialize gemm arguments. This should help reduce overhead a bit. The only notable API change exposed to the user is that all grouped gemm functions now return lists of outputs where bf16 previously returned a single tensor blob. This does mean that in some cases we'll have to do an extra `torch.stack` to unify the groups. If this turns out to be costly, I think we can instead have two grouped gemm implementations, one for dynamic (which returns a single tensor) and one for static shapes which returns a list of tensors. Differential Revision: D67423469 Reviewed By: jianyuh
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
jwfromm
force-pushed
the
export-D67261862
branch
from
26ab8e8 to
f2823db
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
jwfromm
pushed a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Differential Revision: D67261862 Reviewed By: zjing14
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
jwfromm
force-pushed
the
export-D67261862
branch
from
f2823db to
df2c145
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
jwfromm
force-pushed
the
export-D67261862
branch
from
df2c145 to
825c2cd
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
jwfromm
added a commit
to jwfromm/FBGEMM
that referenced
this pull request
Dec 31, 2024
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
jwfromm
force-pushed
the
export-D67261862
branch
from
825c2cd to
fe36ce2
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
Summary: Pull Request resolved: pytorch#3526 X-link: facebookresearch/FBGEMM#608 Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :( Reviewed By: zjing14 Differential Revision: D67261862
jwfromm
force-pushed
the
export-D67261862
branch
from
fe36ce2 to
92e2946
Compare
|
This pull request was exported from Phabricator. Differential Revision: D67261862 |
|
This pull request has been merged in 4c0d4f7. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: Implementation of CK based BF16 Grouped Gemm. Currently performance is quite poor :(
Reviewed By: zjing14
Differential Revision: D67261862