Is there any way to extract lora adapter from merged model?

[tjtanaa]

Is there any way to extract lora adapter from merged model?

Recently, there have been a marvelous improvement in deploying multiple LoRA. I wonder is there any existing code that could be use to extract the merged weights back into LoRA adapter to make use of the existing multi lora inference engine, e.g. lorax and storage.

[younesbelkada]

Technically I think this is possible, you can merge adapters that you want to merge by specifying adapter_names in merge:

peft/src/peft/tuners/lora/layer.py

Line 267 in bffbbbf

def merge(self, safe_merge: bool = False, adapter_names: Optional[List[str]] = None) -> None:

- after that you can have access to the list of merged adapters inside the merged_adapters attribute of LoraLayer: https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/layer.py#L48

You can simply use that list to retrieve LoRA weights from lora_A and lora_B module dicts as their weights will still remain there after merging (and not unloading). If you merge + unload the model it might be complex to retrieve the merged lora weights

[BenjaminBossan]

To add to Younes' response.

to extract the merged weights back into LoRA adapter to make use of the existing multi lora inference engine

If your use case is that you have multiple LoRA adapters and you want to merge adapter 1 for faster inference, but later switch to adapter 2 and merge that, later to adapter 3 etc. here is how I would do it.

1. Load all adapters. At this point, adapter 1 (the first one loaded) should be the active_adapter.
2. Call merge_adapter, which should merge the active adapter, in this case 1 (don't use merge_and_unload, as Younes mentioned)
3. Once you're finished with that adapter, call unmerge_adapter
4. Switch to adapter 2 by calling set_adapter; go back to step 2.

And for future reference, the permalink of Younes' link:

peft/src/peft/tuners/lora/layer.py

Line 48 in c22a8e5

self.merged_adapters = []

[tjtanaa]

If you merge + unload the model it might be complex to retrieve the merged lora weights

What I am currently looking for is this option as there are many models e.g. WizardLM, dolphine, openhermes, etc are all models with merged lora weights (merge + unload). I am interesting in this as s-lora and lora-x helps to schedule how the lora adapters will be used, and I only need to host one base model and multiple lora (which takes up significantly smaller amount of GPU memory).

[BenjaminBossan]

As Younes mentioned, if the adapters are completely merged into the base model weights, it's not trivial to extract them again. There is nothing in PEFT that supports this, you would have to do this manually, and it would require access to both the merged weights + adapter weights separately, or the merged weights + base weights separately.

[tanyav2]

You could use get_peft_model_state_dict to extract the adapter weights. It's also discussed in PEFT low level API.

peft/src/peft/utils/save_and_load.py

Lines 41 to 43 in 26504a0

	def get_peft_model_state_dict(
	model, state_dict=None, adapter_name="default", unwrap_compiled=False, save_embedding_layers="auto"
	):

[thomasgauthier]

I implemented something in this direction using singular value decomposition (SVD). I call it LoRD for Low-Rank Decomposition