📖A curated list of Awesome LLM Inference Papers with codes. 🎉🎉

📚 Tensor/CUDA Cores, 📖150+ CUDA Kernels, toy-hgemm library🔥(achieve the performance of cuBLAS 🎉🎉).

Shush is an app that deploys a WhisperV3 model with Flash Attention v2 on Modal and makes requests to it via a NextJS app

Triton implementation of FlashAttention2 that adds Custom Masks.

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

Flash Attention Implementation with Multiple Backend Support and Sharding This module provides a flexible implementation of Flash Attention with support for different backends (GPU, TPU, CPU) and platforms (Triton, Pallas, JAX).

Uses the powerful WhisperS2T and Ctranslate2 libraries to batch transcribe multiple files

Poplar implementation of FlashAttention for IPU

Toy Flash Attention implementation in torch

Transcribe audio in minutes with OpenAI's WhisperV3 and Flash Attention v2 + Transformers without relying on third-party providers and APIs. Host it yourself or try it out.