Get up and running with Llama 3.3, Phi 4, Gemma 2, and other large language models.

LLM inference in C/C++

Distribute and run LLMs with a single file.

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains

Home of StarCoder2!

Self-hosted AI coding assistant

Use Codestral Mamba with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.

Artificial Intelligence Infrastructure-as-Code Generator.

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU su…

A high-throughput and memory-efficient inference and serving engine for LLMs