Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TriePagedAttentionCache #632

Merged
merged 23 commits into from
Dec 4, 2024
Merged

TriePagedAttentionCache #632

merged 23 commits into from
Dec 4, 2024

Commits on Dec 4, 2024

  1. add unit tests and fix numerous small problems

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    a1946b8 View commit details
    Browse the repository at this point in the history

  2. all tests passing

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    8437a7f View commit details
    Browse the repository at this point in the history

  3. all but publishing working

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    24a4313 View commit details
    Browse the repository at this point in the history

  4. naming consistency Page -> Paged

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    5c87b2b View commit details
    Browse the repository at this point in the history

  5. passing all tests now

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    07d3040 View commit details
    Browse the repository at this point in the history

  6. name and documentation update publish_pages -> publish_pages_for_tokens

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    8f83118 View commit details
    Browse the repository at this point in the history

  7. undo accidental edit of messages.py

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    5c37666 View commit details
    Browse the repository at this point in the history

  8. remove not-very-useful test case

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    802c4e2 View commit details
    Browse the repository at this point in the history

  9. fix missing rename

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    3c7cc6c View commit details
    Browse the repository at this point in the history

  10. use trie by default

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    f59485e View commit details
    Browse the repository at this point in the history

  11. add errmsg for unknown kvcache type

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    646c167 View commit details
    Browse the repository at this point in the history

  12. fix typoes

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    5801d27 View commit details
    Browse the repository at this point in the history

  13. trie doesn't work yet reverting to base for default

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    7f6e5a3 View commit details
    Browse the repository at this point in the history

  14. rename cache_type to prefix_sharing_algorithm

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    ca29397 View commit details
    Browse the repository at this point in the history

  15. add another test case for trie and xfail it

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    7abaa7c View commit details
    Browse the repository at this point in the history

  16. fix xpass by fixing a place where i forgot to pass prefix_sharing_alg… 

    …orithm in conftest.py
    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    1e35634 View commit details
    Browse the repository at this point in the history

  17. organize test cases

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    1a419f1 View commit details
    Browse the repository at this point in the history

  18. add an extend_allocation method to cache allocations and make decode … 

    …requests use that instead
    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    85bc5fc View commit details
    Browse the repository at this point in the history

  19. remove xfail because we are now PASSING!

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    feb3aa9 View commit details
    Browse the repository at this point in the history

  20. fix typo; force certain args to be kwargs

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    7e07cc2 View commit details
    Browse the repository at this point in the history

  21. use a reference counter class to replace int for ref_count in trie node

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    d71826f View commit details
    Browse the repository at this point in the history

  22. fixed problem with refcount not having a default

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    871b622 View commit details
    Browse the repository at this point in the history

  23. use integer ops instead of float with ceil / floor

    @renxida
    renxida committed Dec 4, 2024
    Configuration menu
    Copy the full SHA
    c32be80 View commit details
    Browse the repository at this point in the history