Home Creators Posts Import Register Favorites Logout

Content

Huge news for Kohya GUI - Now you can fully Fine Tune / DreamBooth FLUX Dev with as low as 6 GB GPUs without any quality loss compared to 48 GB GPUs - Moreover, Fine Tuning yields better results than any LoRA training could

Config Files

I published all configs here : https://www.patreon.com/posts/112099700

Tutorials

Fine tuning tutorial in production

Windows FLUX LoRA training (fine tuning is same just config changes) : https://youtu.be/nySGu12Y05k

Cloud FLUX LoRA training (RunPod and Massed Compute ultra cheap) : https://youtu.be/-uhL2nW7Ddw

LoRA Extraction

The checkpoint sizes are 23.8 GB but you can extract LoRA with almost no loss quality - I made a research and public article / guide for this as well

LoRA extraction guide from Fine Tuned checkpoint is here : https://www.patreon.com/posts/112335162

Info

This is just mind blowing. The recent improvements Kohya made for block swapping is just amazing.

Speeds are also amazing that you can see in image 2 - of course those values are based on my researched config and tested on RTX A6000 - same speed as almost RTX 3090

Also all trainings experiments are made at 1024x1024px. If you use lower resolution it will be lesser VRAM + faster speed

The VRAM usages would change according to your own configuration - likely speed as well

Moreover, Fine Tuning / DreamBooth yields better results than any LoRA could

Installers

1-Kohya GUI accurate branch and Windows Torch 2.5 Installers and test prompts shared here : https://www.patreon.com/posts/110879657

The link of Kohya GUI with accurate branch : https://github.com/bmaltais/kohya_ss/tree/sd3-flux.1

Files

Comments

Lukas

Could you create a comprehensive tutorial on how to make a style using Dreambooth?

Bartosz Polecki

Hi Mate, i have unrelated question, can you update the template for massed compute A6000 to include flux model, VAE and t5 already downloaded ? I tend to generate stuff remotely during pause in the work but because i have to download models every time i deploy new gpu everything for flux have to be downloaded all over again and it takes my pause away ;-) Regards

cool1

Would the dreambooth & loras also train at higher resolution than 1024x1024 for flux, now that the GPU requirements are less? Or will it only work properly/at high quality up to 1024x1024? Or would it slow down too much if done higher?

Steve

This is awesome

Davis

Dude absolutely amazing results look incredible. Looking forward to trying

JeroenvanH

Hi, I just tried the latest install (v40). 2 things: I have python 3.12 installed so I need to stop the setup.bat from running to force a 3.10 install. Maybe that's something you can incorporate in your install script? After I install torch 2.5 and do gui.bat it says that the wrong torch is installed and reverts back to 2.4. Any ideas on that?

SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts

well normally i make python 3.10 to be used but in this case we are using official installer of bmaltais kohya so he needs to make it. you need to use python 3.10 , so make it default while installing . yes use my bat file to start it will skip that return back

JeroenvanH

I will look for your startup bat, thanks! And on the python version: There are 2 installers in the official kohya directory. A standard one, that just takes the default python to setup the venv (in my case 3.12), but there's also an installer that specifically uses 3.10. Not sure why bmaltais didn't make that 3.10 installer the default one, because it won't run on 3.12.

Basrad Wong

How do you resume training Dreambooth Flux after the training crashes?

Jan Zhor

it flies 100MB/s on massed compute, so no big deal, everything ready in few minutes

Pew

Doc, absent explicit detail, do you know what might be causing my RTX 4090 to spill over to system ram at high levels? I tried 24GB_GPU_23150MB_10.2_second_it_Tier_1.json, though after coming across the issue, I also tried the 16GB .json and in that test, it wouldn't use all of the available VRAM and instead used system memory 3x the VRAM it was using. I was trying to run 80 epochs, 1 base image repeat for 38 training images which lead to around 3,040 steps or so - nothing too taxing I thought. I have 128 GB System Ram.

Pew

I'm using the base 24GB_GPU_23150MB_10.2_second_it_Tier_1.json At the moment, it shows; steps: 26%|███▊ | 802/3040 [3:22:32<9:25:12, 15.15s/it, avr_loss=0.418] Can you please look over my json as configured for this training in session? https://codeshare.io/r4rvRz I've also posted the cmd.exe output which has some errors, can you look this over? https://codeshare.io/dejYRR I don't quite understand - and in all, it seems slower than anticipated, and right now, the training has been running for 4hrs 5min, and Task Manager is showing: GPU Utilization: Bouncing dramatically, and randomly between, 29%, 99%,100%,88%,76%,53%,and so on, as this is new to me and I've not seen that kind of activity before. Dedicated GPU Memory: Also shows active fluctuation from 21.0 GB to 23.3 GB of GPU VRAM / 24.0 GB. Shared GPU Memory: Rather consistent, at 8.8 / 63.9 GB GPU Memory: Shows 32.1 / 87.9 GB with minimal fluctuation. The Windows 11 machine itself has 128 GB System memory installed with Intel(R) Core(TM) i9-14900KF 3.20 GHz processor. /it increased by one second in under 225 steps: steps: 34%|██▍ | 1025/3040 [4:44:13<9:18:44, 16.64s/it, avr_loss=0.419] I welcome your insight and feedback, please and thank you!

Pew

I spun up a Runpod instance, using "1 x L40S 32 vCPU 125 GB RAM," and the same dataset and while the json is different (24GB vs 48GB), I used the same hyper-parameters I had on the RTX 4090 and now this training in progress with the 48 GB GPU is showing steps: 1%|▌ | 22/3040 [01:48<4:08:05, 4.93s/it, avr_loss=0.395

JeroenvanH

It worked, thank you! Trained my first model. As you said: Much better quality than a LORA. About 10s/it on my 3090, so it does take a long time. BTW, love your prompts in the series you posted here!

Pew

Update: Hours later and the RTX 4090 continues to slow down, yet I'm not using the computer for anything else. I suspect there remains an issue with my setup. steps: 77%|████▉ | 2329/3040 [13:41:02<4:10:38, 21.15s/it, avr_loss=0.421]

Leilu Dallas

Yo, I can't make it to work on my 4070 12GB and 32GB ram : OOM error. I tested 12GB_GPU_10800MB_17.2_second_it_Tier_1.json, Rank_6_10500MB_10_63_Second_IT.json and Rank_5_11700MB_12_40_Second_IT.json

AI Hotshots - time to get curious (edited)

Comment edits

2026-09-07 10:00 Hi, I tried now Rank_3_T5_XXL_23500MB_11_35_Second_IT on my 4090 on latest kohya_ss etc. Trains lora very slow and feeled endless. Currently with 13,90s / it...I think to cancel the training of ca. 180 picture. Any tip for improvement ?
2024-10-10 11:04

Bartosz Polecki

Ok,thanks for the "button" Mate its good enough. Regards !

J.ruthless

Hi I followed instructions, I have 4090, 32gb ram, and when I run 24,16,or 12 gb config my completion time is always around 22000.00.00 hours , I do have accurate Cuda , git and python installed. I did run flux fix and also manually answered questions for choice 5 of install.

Bartosz Polecki

Nah everything's great, efficiency went up 100% ;-)

N V

I am also having this issue; rtx4090; starting from last week.

Leilu Dallas

That was it. With 64GB of ram, I don't have any more problems. https://i.imgur.com/xNO7epE.png I have a question in passing: why do my 22 images turn into 58 examples? Is it because they're not squares? I want to train some portraits of an old CRPG, but as well as being taller than they are wide, some of them are of rather low quality (582px*900px). Is this a problem? Thanks

Pew

Thank you! I discovered by problem, I believe. Same config, but using 25 base training images vs the count I had tried previously which was over 160 but under 200 images, I don't recall. I will have to dial this in but clearly the config I was using had too many images I think.

Pew

I think I discovered by problem, I believe. Same config, but using 25 base training images vs the count I had tried previously which was over 160 but under 200 images, I don't recall. I will have to dial this in but clearly the config I was using had too many images. Could that have been the cause? Now I am actively getting 6.99s/it, avr_loss=0.396] at the moment.