This repository provides a collection of open-source machine learning language models (LLMs) that are licensed for commercial use (e.g., Apache 2.0, MIT, OpenRAIL-M). Contributions are welcome!
| Language Model | Release Date | Checkpoints | Article/Blog | Parameters (B) | Context Length | License | Try it |
|---|---|---|---|---|---|---|---|
| T5 | 2019/10 | T5 & Flan-T5, Flan-T5-xxl (HF) | Exploring the Bounds of Transfer Learning with a Unified Text-to-Text Transformer | 0.06 - 11 | 512 | Apache 2.0 | T5-Large |
| UL2 | 2022/10 | UL2 & Flan-UL2, Flan-UL2 (HF) | UL2 20B: An Open-Source Unified Language Learner | 20 | 512, 2048 | Apache 2.0 | |
| Cerebras-GPT | 2023/03 | Cerebras-GPT | Cerebras-GPT: A Family of Open, Compute-Efficient, and Large Language Models (Article) | 0.111 - 13 | 2048 | Apache 2.0 | Cerebras-GPT-1.3B |
| Open Assistant (Pythia Family) | 2023/03 | OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1 | Democratizing Large Language Model Alignment | 12 | 2048 | Apache 2.0 | Pythia-2.8B |
| Pythia | 2023/04 | [pythia 70M - 12B](https://github.com/EleutherAI/pythia |
) | Pythia: A Suite for Analyzing Large Language Models during Training and Scaling | 0.07 - 12 | 2048 | Apache 2.0 | | | Dolly | 2023/04 | dolly-v2-12b | Free Dolly: Introducing the First Open and Commercially Viable Fine-Tuned Language Model for Real Instructions | 3, 7, 12 | 2048 | MIT | | | DLite | 2023/05 | dlite-v2-1_5b | Announcing DLite V2: Lightweight Open LLMs that Can Run Anywhere | 0.124 - 1.5 | 1024 | Apache 2.0 | DLite-v2-1.5B | | RWKV | 2021/08| RWKV, ChatRWKV | The RWKV Language Model (and My LLM Tricks) | 0.1 - 14 | Infinite (RNN) | Apache 2.0 | | | GPT-J-6B | 2023/06 | GPT-J-6B, GPT4All-J | GPT-J-6B: A 6B JAX-based Transformer Language Model | 6 | 2048 | Apache 2.0 | | | GPT-NeoX-20B | 2022/04 | GPT-NEOX-20B | GPT-NeoX-20B: An Open-Source Autoregressive Language Model | 20 | 2048 | Apache 2.0 | | | Bloom | 2022/11 | Bloom | BLOOM: An Open-Access Multilingual Language Model with 176 Billion Parameters | 176 | 2048 | OpenRAIL-M v1 | | | StableLM-Alpha
| 2023/04 | StableLM-Alpha | Stability AI Launches the First Model of Its StableLM Language Model Suite | 3 - 65 | 4096 | CC BY-SA-4.0 | | | FastChat-T5 | 2023/04 | FastChat-T5: Our compact and commercially compatible chatbot is now available. | fastchat-t5-3b-v1.0 | 3 | 512 | Apache 2.0 | | | h2oGPT | 2023/05 | Building the World's Best Open-Source Large Language Model: H2O.ai's Journey | h2oGPT | 12 - 20 | 256 - 2048 | Apache 2.0 | | | MPT-7B | 2023/05 | MPT-7B | ?? | ? | ? | ? | | | ANIMALITY.ai LLM | ??? | ??? | ??? | 8.8 Million | 8000 | ??? | ??? |
Please note that the ANIMALITY.ai LLM with 8.8 Million parameters and 8K token size is not listed in the original table.