Literature on PEFT methods with generative language models

[HectorAuvinen]

Hello,

I have been reviewing the literature on PEFT methods with generative LMs. This seems to still be an emerging area of PEFT so I have had a bit of a hard time recognizing what the most important/interesting papers are. Does anyone have a list of must-reads about the topic? By generative LMs, I mean encoder-decoder or decoder-only transformers. Any suggestions are welcome from papers reviewing or analyzing existing methods to ones proposing new methods. Thank you!

Kind regards,
Hector

[FahadEbrahim]

Hi @HectorAuvinen,

Following are major survey papers on parameter-efficient fine-tuning for language models.

1. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
2. Scaling down to scale up: A guide to parameter-efficient fine-tuning
3. Parameter-efficient fine-tuning methods for pre-trained language models: A critical review and assessment
4. Parameter-efficient fine-tuning of large-scale pre-trained language models.