Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[New feature] Add inference load balance controller for fastdeploy llm #2276

Open
wants to merge 9 commits into
base: llm
Choose a base branch
from
Open

[New feature] Add inference load balance controller for fastdeploy llm #2276

wants to merge 9 commits into from

Commits on Nov 8, 2023

  1. add inference load balancer for fastdeploy llm

    @rainyfly
    rainyfly committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    84bb411 View commit details
    Browse the repository at this point in the history

  2. add inference load balance controller for llm

    @rainyfly
    rainyfly committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    1d5f0e8 View commit details
    Browse the repository at this point in the history

  3. add ic for llm

    @rainyfly
    rainyfly committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    7c93c59 View commit details
    Browse the repository at this point in the history

  4. add ic for llm

    @rainyfly
    rainyfly committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    ad0b0e5 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2023

  1. add fastdeploy ic for llm

    @rainyfly
    rainyfly committed Nov 20, 2023
    Configuration menu
    Copy the full SHA
    5b60c20 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. add fastdeploy ic to llm

    @rainyfly
    rainyfly committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    7efb041 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Fix asyncio.CancelError exception

    @rainyfly
    rainyfly committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    eb08074 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. do not block ic by same req_id

    @rainyfly
    rainyfly committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    129537c View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2023

  1. update stability for redis client

    @rainyfly
    rainyfly committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    2b9b3f0 View commit details
    Browse the repository at this point in the history