Skip to content

Commit

Permalink
Added table for RWKV version status
Browse files Browse the repository at this point in the history
  • Loading branch information
PicoCreator authored Dec 9, 2023
1 parent 7336738 commit 20be06c
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,18 @@ So it's combining the best of RNN and transformer - great performance, fast infe
[![RWKV paper cover](./img/RWKV-paper.png)](https://arxiv.org/abs/2305.13048)
- [arXiv (2305.13048) paper](https://arxiv.org/abs/2305.13048)

# Current Version Status

| Version | v4 Raven | v4 World | v5 World | v6 World |
|---|---|---|---|---|
| Paper | 🎓[Paper Accepted @ EMNLP 2023](https://arxiv.org/abs/2305.13048) | (no architecture change) | wip | wip |
| Overall Status | 🌚 EOL - Recommended to use v5 world instead | ✅ GA - Recommended to switch to v5 world when possible | 🔧 Training | 🧪 Prototyping |
| 0.4B model | [Fully Trained : rwkv-pile-430m](https://huggingface.co/RWKV/rwkv-4-430m-pile) |[Fully Trained](https://huggingface.co/RWKV/rwkv-4-world-430m) |[Fully Trained](https://huggingface.co/BlinkDL/rwkv-5-world/blob/main/RWKV-5-World-0.4B-v2-20231113-ctx4096.pth) | 🧪Prototyping |
| 1.5B model | [Fully Trained : rwkv-raven-1b5](https://huggingface.co/RWKV/rwkv-raven-1b5) |[Fully Trained](https://huggingface.co/RWKV/rwkv-4-world-1b5) |[Fully Trained](https://huggingface.co/BlinkDL/rwkv-5-world/blob/main/RWKV-5-World-1B5-v2-20231025-ctx4096.pth) | 🧪Prototyping |
| 3B model | [Fully Trained : rwkv-raven-3b](https://huggingface.co/RWKV/rwkv-raven-3b) |[Fully Trained](https://huggingface.co/RWKV/rwkv-4-world-3b) | 🔧[Finalizing](https://huggingface.co/BlinkDL/rwkv-5-world/blob/main/RWKV-5-World-3B-v2-20231118-ctx16k.pth) | 🧪Prototyping |
| 7B model | [Fully Trained : rwkv-raven-7b](https://huggingface.co/RWKV/rwkv-raven-7b) |[Fully Trained](https://huggingface.co/RWKV/rwkv-4-world-7b) | 🔧[Training in process](https://huggingface.co/BlinkDL/temp/blob/main/rwkv-x052-7b-world-v2-79%25trained-20231208-ctx4k.pth) | |
| 14B model | [Fully Trained : rwkv-raven-14b](https://huggingface.co/RWKV/rwkv-raven-14b) | not-planned | scheduled | |

# TLDR vs Existing transformer models

**Good**
Expand Down

0 comments on commit 20be06c

Please sign in to comment.