Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat (nn/sdpa): quantization of scaled dot-product attention #1090

Open
wants to merge 17 commits into
base: dev
Choose a base branch
from
Open

Feat (nn/sdpa): quantization of scaled dot-product attention #1090

wants to merge 17 commits into from
+365 −5

Commits on Nov 28, 2024

  1. feat nn: add ScaledDotProductAttention and `QuantScaledDotProductAt… 

    …tention` classes
    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    242e15f View commit details
    Browse the repository at this point in the history

  2. Feat (graph/standardize): Add SDPA conversion to modular version

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    5d700e1 View commit details
    Browse the repository at this point in the history

  3. Feat (example/llm): Specify LLMs to use SDPA for their attn implement… 

    …ation
    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    67daa35 View commit details
    Browse the repository at this point in the history

  4. Fix (nn/sdpa): formatting

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    00da24e View commit details
    Browse the repository at this point in the history

  5. Fix (nn/sdpa): Updated argument to match qsdpa

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    0bfbc9d View commit details
    Browse the repository at this point in the history

  6. feat (nn): bugfixes in QuantSDPA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    567d7a9 View commit details
    Browse the repository at this point in the history

  7. Feat (example/llm): Adding functions to replace SDPA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    4fd4e5d View commit details
    Browse the repository at this point in the history

  8. feat (example/generative): Replace SDPA with QuantSDPA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    2f3adb8 View commit details
    Browse the repository at this point in the history

  9. feat (example/llm): Add argument to quantize SDPA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    0bfb838 View commit details
    Browse the repository at this point in the history

  10. test (llm/sdpa): Added basic tests for SDPA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    d88a05c View commit details
    Browse the repository at this point in the history

  11. Fix (example/llm): workaround for new OPT default attention

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    70397a1 View commit details
    Browse the repository at this point in the history

  12. Fix (graph): Changed SDPA import

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    480cca2 View commit details
    Browse the repository at this point in the history

  13. Fix (graph): Removed SDPA from default standardize script.

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    a0a7a92 View commit details
    Browse the repository at this point in the history

  14. test (nn/sdpa): Added basic sanity check test

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    ef1b779 View commit details
    Browse the repository at this point in the history

  15. Fix (nn): Fix in QSPDA

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    e3d3c4a View commit details
    Browse the repository at this point in the history

  16. test (nn): Added quant_disabled QSPDA test

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    3798923 View commit details
    Browse the repository at this point in the history

  17. test (fix): sdpa import

    @nickfraser
    nickfraser committed Nov 28, 2024
    Configuration menu
    Copy the full SHA
    6b1e51a View commit details
    Browse the repository at this point in the history