Flux LoRA Training Setting

[matabear-wyx]

Hi, thank you for this fantastic repository! I've been using it for my Flux LoRA training for a couple of weeks now, and it has been working great.

I recently noticed that you made significant changes to the Flux Quickstart documentation. I decided to try out the new settings, but in my case, I found that the old recommendations worked better for me. Specifically, I’m training a LoRA for person images, and with the previous settings, my LoRA converged in around 1500 steps, producing excellent results.

I was wondering if we could discuss this further? It might be helpful to understand the rationale behind the new settings and whether the previous approach is still valid for certain cases. (In my case, I decide to continue with the previous settings)

Thanks again for your amazing work!

[bghira]

can you elaborate on the differences in the configuration?

[matabear-wyx]

at least for person lora training：
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --base_model_precision=int8-quanto"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --text_encoder_1_precision=no_change --text_encoder_2_precision=no_change"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --lora_rank=16"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --max_grad_norm=1.0"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --base_model_default_dtype=bf16"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --flux_lora_target=all"
export TRAINER_EXTRA_ARGS="${{TRAINER_EXTRA_ARGS}} --lora_init_type=loftq"
export OPTIMIZER="adamw_bf16"

export LEARNING_RATE=9e-5
export RESOLUTION=512
export MINIMUM_RESOLUTION=$RESOLUTION
export LR_SCHEDULE="constant"

works very good for me

[bghira]

well, that is a PEFT LoRA. it has had problems in the past with resuming training and still has issues with multigpu training when quantised.

LyCORIS LoKr is superior to PEFT LoRA, it just needs a higher learning rate. so chances are you will be happier with the new defaults. you just need to experiment a bit.

[matabear-wyx]

Since the resume training problem was solved with the latest update of optimum-quanto, and I don't need multigpu training, I assume I can continue with the previous training scripts since it only uses 13G VRAM and can produce good enough results in my scenario.

[bghira]

see this report I prepared on some of the benefits of LoKr and how to get the most out of it.