How to continue a training from a LoRA checkpoint ?

[l-k-11235]

I tried to relaunch the finetuning with the same config but it restarted the training from scratch. Moreover, as the checkpoint directory only contains the lora weights, it may require some adaptations.

[vince62s]

You cannot do this. You will need to merge your lora weight first and then train_from the merged model.

[l-k-11235]

I have tried but it does not perform well (see #64 ).
Moreover it is quite different, as the LoRA matrixes and the optimizer states will be reset.

[vince62s]

There is no other path at the moment and doubtful about the other one to be implemented.