Just want to say a MASSIVE thank you!

[fahadshery]

I have been trying to get Flux working using comfyui.
your project works but it's slow!
Here are the logs:

got prompt
Using xformers attention in VAE
Using xformers attention in VAE
model weight dtype torch.bfloat16, manual cast: torch.float32
model_type FLUX
Requested to load FluxClipModel_
Loading 1 new model
loaded completely 0.0 9319.23095703125 True
clip missing: ['text_projection.weight']
Requested to load Flux
Loading 1 new model
loaded partially 17764.8386796875 17760.509887695312 0
100%|██████████| 20/20 [23:09<00:00, 69.50s/it]
Requested to load AutoencodingEngine
Loading 1 new model
loaded completely 0.0 319.7467155456543 True
Prompt executed in 2506.89 seconds

I am using Nvidia Tesla P40 24G card ...
I had to pass the --disable-cuda-malloc in the CLI_ARGS otherwise I was getting some sampler error. (See here: comfyanonymous/ComfyUI#1845)

thank you for your efforts...

(I am using cu121-megapack) because my CUDA version is 12.2...

[YanWenKun]

Glad you made it!

In Flux cases, I strongly recommend updating your GPU driver to the latest possible. Then using CUDA 12.4 + PyTorch 2.4, they will do the magic.

Check out the ComfyUI Examples and see if Schnell fp8 performs better:
https://comfyanonymous.github.io/ComfyUI_examples/flux/

For reference, I'm using a Titan Xp 12GB (similar spec but half vram to P40). Passing --fast --use-pytorch-cross-attention to CLI_ARGS (which means not using xFormers), and it can run the FLUX1.Schnell FP8 example workflow in ~108 seconds.

Good luck!

Edit (translated by chatgpt):

I suggest that you identify the bottleneck, such as slow disk I/O, low memory, frequent paging, or potential workflow issues.

Furthermore, I tested the example workflow more thoroughly. Based on the time reported as "Prompt executed in ??? seconds," the results were as follows:

• On Linux, regardless of whether using torch 2.4.0/2.4.1, --fast on/off, xformers on/off, or custom nodes on/off, the execution time remained consistently around 108 seconds.
• On Windows, using my portable package, the execution time stabilized at 69 seconds, regardless of whether --fast was enabled or not.

I suspect this is related to GPU drivers, as the behavior differs between SD1.5 and SDXL generation. If possible, try using Flux on Windows to see if there's any performance difference.