From 3964f74583cf6c78ec42f994da7e33703531f62b Mon Sep 17 00:00:00 2001 From: Eugene Cheah Date: Thu, 21 Sep 2023 11:08:59 -0700 Subject: [PATCH] emphasis on compute savings --- docs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 35f5f82..fcce800 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,7 +18,7 @@ So it's combining the best of RNN and transformer - great performance, fast infe # TLDR vs Existing transformer models **Good** -+ Lower resource usage (VRAM, CPU, GPU, etc) when running and training ++ Lower resource usage (VRAM, CPU, GPU, etc) when running and training. With 10x to a 100x lower compute requirements compared to transformers with large context sizes. + Scales to any context length linearly (transformers scales quadratically) + Perform just as well, in terms of answer quality and capability + RWKV models are generally better trained in other languages (e.g. Chinese, Japanese, etc), then most existing OSS models