Comprehensive Different Version and Precision FLUX Models Speed and VRAM Usage Comparison (Patreon)

I just updated the automatic FLUX models downloader scripts with newest models and features. Therefore I decided to test all models comprehensively with respected to their peak VRAM usage and also their image generation speed.

Automatic downloader scripts : https://www.patreon.com/posts/109289967

 Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.

Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord

Please also Star, Watch and Fork our Stable Diffusion & Generative AI  GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)

Testing Results

All tests are made with 1024x1024 pixels generation, CFG 1, no negative prompt

All tests are made with latest version of SwarmUI (0.9.2.1)

These results are not VRAM optimized - fully loaded into VRAM and thus maximum speed

All VRAM usages are peak which happens when finally decoding with VAE after all steps completed

Below tests are on A6000 GPU on massed Compute with FP8 T5 text encoder - default

Full tutorial for how to use locally (on your PC on Windows) and on Massed Compute (31 cents per hour for A6000 GPU) is at below

SwarmUI full public tutorial : https://youtu.be/bupRePUOA18

Testing Methodology

Tests are made on a cloud machine thus VRAM usages were below 30 mb before starting SwarmUI

nvitop library is used to monitor VRAM usages during generation and peak VRAM usages recorded which usually happens when VAE decoding image after all steps completed

SwarmUI reported timings are used

First generation never counted, always multiple times generated and last one used

Below Tests are Made With Default FP8 T5 Text Encoder

flux1-schnell_fp8_v2_unet 

Turbo model FP 8 weights (model only 11.9 GB file size)

19.33 GB VRAM usage - 8 steps - 8 seconds

flux1-schnell 

Turbo model FP 16 weights (model only 23.8 GB file size)

Runs at FP8 precision automatically in Swarm UI

19.33 GB VRAM usage - 8 steps - 7.9 seconds

flux1-schnell-bnb-nf4 

Turbo 4bit model - reduced quality but VRAM usage too

Model + Text Encoder + VAE : 11.5 GB file size

13.87 GB VRAM usage - 8 steps - 7.8 seconds

flux1-dev

Dev model - Best quality we have

FP 16 weights - model only 23.8 GB file size

Runs at FP8 automatically in Swarm UI

19.33 GB VRAM usage - 30 steps - 28.2 seconds

flux1-dev-fp8

Dev model - Best quality we have

FP 8 weights (model only 11.9 GB file size)

19.33 GB VRAM usage - 30 steps - 28 seconds

flux1-dev-bnb-nf4-v2

Dev model - 4 bit model - slightly reduced quality but VRAM usage too

Model + Text Encoder + VAE : 12 GB file size

14.40 GB - 30 steps - 27.25 seconds

FLUX.1-schnell-dev-merged

Dev + Turbo (schnell) model merged

FP 16 weights - model only 23.8 GB file size

Mixed quality - Requires 8 steps

Runs at FP8 automatically in Swarm UI

19.33 GB - 8 steps - 7.92 seconds

Below Tests are Made With Default FP16 T5 Text Encoder

FP16 Text Encoder slightly improves quality and also increases VRAM usage

Below tests are on A6000 GPU on massed Compute with FP16 T5 text encoder - If you overwrite previously downloaded FP8 T5 text encoder (automatically downloaded) please restart SwarmUI to be sure

Don't forget to select Preferred DType to set FP16 precision - shown in tutorial : https://youtu.be/bupRePUOA18

Currently BNB 4bit models are ignoring FP16 Text Encoder and using embedded FP8 T5 text encoders

flux1-schnell_fp8_v2_unet

Model running at FP8 but Text Encoder is FP16

Turbo model : 23.32 GB VRAM usage - 8 steps - 7.85 seconds

flux1-schnell

Turbo model - DType set to FP16 manually so running at FP16

34.31 GB VRAM - 8 steps - 7.39 seconds

flux1-dev

Dev model - Best quality we have

DType set to FP16 manually so running at FP16

34.41 GB VRAM usage - 30 steps - 25.95 seconds

flux1-dev-fp8

Dev model - Best quality we have

Model running at FP8 but Text Encoder is FP16

23.38 GB - 30 steps - 27.92 seconds

My Suggestions and Conclusions

If you have a GPU that has 24 GB VRAM use flux1-dev-fp8 and 30 steps

If you have a GPU that has 16 GB VRAM use flux1-dev-bnb-nf4-v2 and 30 steps

If you have a 12 GB VRAM or below GPU use flux1-dev-bnb-nf4-v2 - 30 steps

If it becomes too long to generate images due to your low VRAM, use flux1-schnell-bnb-nf4 and 4 to 8 steps depending on speed and duration that you can wait

FP16 Text Encoder slightly increases quality so 24 GB GPU owners can also use FP16 Text Encoder + FP8 models

SwarmUI is currently able to run FLUX as low as 4 GB GPUs with all kind of optimizations (fully automatic). I even saw someone generated image with 3 GB GPU

I am looking for BNB NF4 version of FLUX.1-schnell-dev-merged model for low VRAM users but couldn't find yet

Hopefully I will update auto downloaders once I got 4bit version of merged model