AdaLoRA without update_and_allocate

[h4iku]

The documentation states that "AdaLoRA has an update_and_allocate() method that should be called at each training step to update the parameter budget and mask, otherwise the adaptation step is not performed." I fine-tuned a model using AdaLoRA and Trainer without performing the update_and_allocate() step, but the resulting model is still quite useful. Now I'm wondering, what should I call this since it is not a proper AdaLoRA-trained model?
Is it just LoRA that targets all linear layers and uses init_r as its r? or is it something else?

[BenjaminBossan]

I'm not sure if this is exactly the same as a LoRA model, as AdaLoRA does a few other things differently. But I would suggest to train LoRA a model with the same settings as you just tried, I would expect it to perform equally well.