Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compressed tensors w8a8 tpu #10435

Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft

Compressed tensors w8a8 tpu #10435

wants to merge 34 commits into from
+515 −197

Commits on Oct 11, 2024

  1. w8a8 working

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    3b0c8a6 View commit details
    Browse the repository at this point in the history

  2. format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    36fc1db View commit details
    Browse the repository at this point in the history

  3. added all kernels

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d83c04c View commit details
    Browse the repository at this point in the history

  4. format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    af9d0f4 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. working on cuda

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    0f9fd21 View commit details
    Browse the repository at this point in the history

  2. added mixed precision directory

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    7b3203f View commit details
    Browse the repository at this point in the history

  3. formatting

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    bf50fa4 View commit details
    Browse the repository at this point in the history

  4. cache current state - w8a16 running oom

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    226ef52 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. [TPU] Ensure torch._sync(param) is called after param.data.copy_()

    @WoosukKwon
    WoosukKwon committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    bb7c741 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. yapf

    @WoosukKwon
    WoosukKwon committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    cf842bd View commit details
    Browse the repository at this point in the history

  2. [TPU] Correctly profile peak memory usage

    @WoosukKwon
    WoosukKwon committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    67039bc View commit details
    Browse the repository at this point in the history

  3. Upgrade PyTorch XLA

    @WoosukKwon
    WoosukKwon committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    0695f77 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into tpu-peak-mem

    @WoosukKwon
    WoosukKwon committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    11cf82f View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2024

  1. stash

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    e016e38 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into compressed-tensors-tpu

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    717b859 View commit details
    Browse the repository at this point in the history

  3. proper merge

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    c848735 View commit details
    Browse the repository at this point in the history

  4. add mixed precision

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    1539915 View commit details
    Browse the repository at this point in the history

  5. format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    f00412a View commit details
    Browse the repository at this point in the history

  6. stash

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    b0a6b70 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'tpu-peak-mem' into compressed-tensors-tpu

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    e812d7e View commit details
    Browse the repository at this point in the history

  8. stash

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    764dda1 View commit details
    Browse the repository at this point in the history

  9. remove name

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    87b2ae6 View commit details
    Browse the repository at this point in the history

  10. revert woosuk change

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    e813ff8 View commit details
    Browse the repository at this point in the history

  11. format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    8cfaa1b View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. update

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    bbc9741 View commit details
    Browse the repository at this point in the history

  2. fix nit

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    eb3f39e View commit details
    Browse the repository at this point in the history

  3. update

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    bb2fbe1 View commit details
    Browse the repository at this point in the history

  4. fix spurious

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    14ccb90 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. stash branch for brittany

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    4092be2 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. merged

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    0e60ae8 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    90e4de8 View commit details
    Browse the repository at this point in the history

  2. cleanup

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    7e6f67f View commit details
    Browse the repository at this point in the history

  3. update format

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    36973d8 View commit details
    Browse the repository at this point in the history

  4. formatting

    @robertgshaw2-neuralmagic
    robertgshaw2-neuralmagic committed Nov 19, 2024
    Configuration menu
    Copy the full SHA
    431a0b4 View commit details
    Browse the repository at this point in the history