From b55dc30ec7395b383adba87b6a0985003461f535 Mon Sep 17 00:00:00 2001 From: Eugene Cheah Date: Fri, 15 Sep 2023 18:19:29 -0700 Subject: [PATCH] finetune adds infctx repo --- docs/advance/finetune.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/advance/finetune.md b/docs/advance/finetune.md index 68deaed..510037c 100644 --- a/docs/advance/finetune.md +++ b/docs/advance/finetune.md @@ -29,6 +29,12 @@ Alternatively, you can use the official repo, to finetune the project without Lo - [https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v4neo](https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v4neo) +Alternatively would be to do finetuning using the infctx trainer + +- [https://github.com/RWKV/RWKV-infctx-trainer/](https://github.com/RWKV/RWKV-infctx-trainer/) + +In general, use the BlinkDL trainer for better raw performance, and use infctx to trade in some speed for infctx size support. + ## Training a model from scratch? Refer to the main project