Skip to content

@vectorch-ai Vectorch

Change the repository type filter

All

    Repositories list

    12 repositories

    • ScaleLLM

      Public
      A high-performance inference system for large language models, designed for production environments.
      performancegpumodelproductioncudaefficiencyinferencetransformerllamaspeculative
      C++
      •
      Apache License 2.0
      •28•377•33•8•Updated Sep 22, 2024Sep 22, 2024

    • whl

      Public
      repository to host python whl package.
      HTML
      •0•0•0•0•Updated Sep 4, 2024Sep 4, 2024

    • flashinfer

      Public
      FlashInfer: Kernel Library for LLM Serving
      Cuda
      •
      Apache License 2.0
      •114•0•0•1•Updated Sep 4, 2024Sep 4, 2024

    • vcpkg

      Public
      C++ Library Manager for Windows, Linux, and MacOS
      CMake
      •
      MIT License
      •6.3k•0•0•0•Updated Jun 22, 2024Jun 22, 2024

    • LLMBench

      Public
      A library for validating and benchmarking LLMs inference.
      Python
      •
      Apache License 2.0
      •1•4•1•0•Updated Jun 18, 2024Jun 18, 2024

    • discussions

      Public
      0•0•0•0•Updated Jun 5, 2024Jun 5, 2024

    • chatbot-ui

      Public
      An open source ChatGPT UI.
      TypeScript
      •
      MIT License
      •7.9k•0•0•0•Updated May 20, 2024May 20, 2024

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1.2k•0•0•0•Updated Oct 15, 2023Oct 15, 2023

    • tokenizers

      Public
      💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
      Rust
      •
      Apache License 2.0
      •778•0•0•0•Updated Aug 4, 2023Aug 4, 2023

    • xformers

      Public
      Hackable and optimized Transformers building blocks, supporting a composable construction.
      Python
      •
      Other
      •600•0•0•0•Updated Aug 1, 2023Aug 1, 2023

    • FasterTransformer

      Public
      Transformer related optimization, including BERT, GPT
      C++
      •
      Apache License 2.0
      •889•0•0•0•Updated Jul 28, 2023Jul 28, 2023

    • ByteTransformer

      Public
      optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
      C++
      •
      Apache License 2.0
      •34•0•0•0•Updated Jul 24, 2023Jul 24, 2023