How to train a custom model

[anhnh2002]

I must take some change llm architecture to adapt with my task.
How to train custom llm using peft?

[BenjaminBossan]

Custom models are generally supported, but you're not providing a lot of information (what adapter, what changes did you make?). Please take a look at our docs on custom models to see if those answer your question.

[anhnh2002]

I add some mlp layer to an LLM. And change training loop, sample data, loss func. Same in this repo https://github.com/nju-websoft/SCKD/tree/main .

[BenjaminBossan]

You can check the default target_module in this file and modify it according to your needs.

Let's use an example. Say your model is gemma and you want to use LoRA. You can see that the entry for gemma in TRANSFORMERS_MODELS_TO_LORA_TARGET_MODULES_MAPPING is ["q_proj", "v_proj"]. Now let's say that your custom MLP module is called my_mlp (i.e that's its attribute name). If you want to apply LoRA to that layer as well, you should set config = LoraConfig(..., target_modules=["q_proj", "v_proj", "my_mlp"]). Alternatively, you can set target_modules="all-linear" to simply modify all nn.Linear layers.

In general, you can just print the model and see all the components and their names. Use this as a guidance for your choice of target_modules.

[anhnh2002]

Got it.
How can i change training loop, sampling data and loss func?

[BenjaminBossan]

PEFT doesn't prescribe any specific training code, use whatever you want. If you have some special things, it's probably easiest to write your own training loop with custom sampler and loss function.

[anhnh2002]

Sorry, my mistake

[anhnh2002]

Please @BenjaminBossan , i have a another question.
For example:
I have a llm with lora like this

from peft import get_peft_config, get_peft_model, LoraConfig, TaskType
model_name_or_path = "bigscience/mt0-large"
tokenizer_name_or_path = "bigscience/mt0-large"

peft_config = LoraConfig(
    task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32, lora_dropout=0.1
)

base_model = AutoModelForSeq2SeqLM.from_pretrained(model_name_or_path)
model = get_peft_model(base_model, peft_config)

I can forward input by model(input). This using base model's params and adapter's params.
How can i forward input without using adapter's params? I want to forward input with and without adapter's params with only one base model loading

[BenjaminBossan]

Use the disable_adapter context like this:

with model.disable_adapter():
    output = model(input)

[anhnh2002]

thank you so much