Flux AI Toolkit preset generate completly noisy image in Comfy UI.

[axel578]

It might not necessarely be a Comfy UI issue since all flux lora training I did on kohya/onetrainer seems to work, and the issue seems to be pretty recent seens it was working with simpletuner before, I used two days old simpletuner version, yet the new version did'nt seem to fix it.

I used your default flux lora training especially with quanto (let me know if its the issue) and use flux target ai-toolkit
Just so you can test it a lora: https://drive.google.com/file/d/1mpFHMTOfbAXVLKKj8A5IfWLr0VIoT32K/view?usp=sharing

[bghira]

ai-toolkit target is not really supported. it is probably safe to remove now, in fact. i would recommend just all or all+ffs for LoRA.

but really LoRA is kinda outdated tech by now. LyCORIS LoKr works much better.

[axel578]

ai-toolkit target is not really supported. it is probably safe to remove now, in fact. i would recommend just all or all+ffs for LoRA.

but really LoRA is kinda outdated tech by now. LyCORIS LoKr works much better.

Yeah, I know the popularity of LyCORIS LoKr and why its know, its very good on paper, but there are many properties (especially on multi resolution) where it doesnt work that well, yet its still pretty good.

Don't remove ai-toolkit, really the more layers the lora is applied on, the easier it is to train a lora on very complex and far away from original learned distribution images, people are judging lora quality only on very easy image dataset (someone's face ...), but they really forget about the capacity of the lora for learning hard styles/image compositons, to be fair, I modified adapter.py to target near every layers of the model and its where I get the best result really.

[bghira]

LoKr has nothing to do with multi-resolution, that's the underlying model at play. unless @KohakuBlueleaf can explain if I am overlooking something.

[KohakuBlueleaf]

Dunno why I see this discussion just now.
But basically yes, all lycoris have nothing to do with multi-res or any other thing
LyCORIS directly do things in nn.module which never care what you are actually doing.

I will say it can be something "overcook" or "over-learn" which may cause forgetting of some good property easier in LyCORIS but that actually means LyCORIS works better LOL.

recommend to adjust lr or factor.