Inquiry about Pruning during Training

[mli-tian]

I have thoroughly reviewed your code but have encountered some confusion. I believe that the prune function in the trainer.py file is responsible for the actual pruning. However, it appears that during the training process, only the mask is modified, and pruning is only performed before inference in the inference.py file. This seems to differ from the description in your paper. Am I misunderstanding something? If my understanding is incorrect, could you please point me to the code that handles pruning during training?

Thank you for your assistance.

[WoutDeRijck]

As is described in Algorithm 1: Progressive pruning with LoRA-guided criterion in the paper. During training the model gets finetuned using lora adapters and a mask is computed. This mask is used during inference to make 50% of the weights 0.