Home
Creators
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register Favorites Keys Review DMs Logout
Community
Telegram ThePornDude
Home Creators Posts Import Register Favorites Logout

The Very Best Kohya GUI Workflow & Config For SD 1.5 Based Models DreamBooth / Full Fine Tuning (Patreon)

Published:
2024-08-11 22:58:00
Edited:
2024-08-12 03:22:00
Imported:
2026-06

Downloads

Content

Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.

Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord

Please also Star, Watch and Fork our Stable Diffusion & Generative AI  GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)

12 August 2024 Update

Scripts were broken, updated and error fixed and tested

Example RunPod path config newest GUI : Example_SD1.5_RunPod_Config.png

I have made over 70 full DreamBooth trainings for over 7 days and meticulously analyzed their results to the find very best training hyper parameters.

120 amazing quality images with their prompt info posted on CivitAI

Part 1 , Part 2 , Part 3 , Part 4 , Part 5 , Part 6 . Each Part has 20 images. You can click (i) icon on images to see their prompts.

We have 3 configs. 

Tier 1 is best quality. Don't use xFormers.

Tier 2 is second best quality. Uses xFormers to reduce VRAM.

xFormers : reduces VRAM, increases speed, reduced quality

Gradient Checkpointing : reduces VRAM, reduces speed, quality sam

All Tier 2 are equal quality and only speed and VRAM usage changes.

Since OneTrainer supports EMA, it is better than Kohya.

OneTrainer config : https://www.patreon.com/posts/97381002

You can download configs:

tier_1_quality_slow_22GB.json

tier_2_lowVRAM_10_2GB.json

tier_2_speed_11_8GB.json

There are 2 strategies of training. Stylized vs Realism.

To find out very best models for both realism and stylization models, I have made 161 models comparison recently if you remember : https://youtu.be/G-oZn4H-aHQ

Models Downloader Script And The Patreon Post Shown In The Video ⤵️ https://www.patreon.com/posts/1-click-download-96666744

1st:

Training for realism. For this training strategy I have chosen the Hyper Realism V3 model from CivitAI. The config file will download it automatically from Hugging Face or alternatively you can give the local path.

2nd:

Training for stylization like 3d render of yourself. For this task I have chosen RealCartoon-Pixar V8 from CivitAI.

To use this model the key change you need to make is, making Clip skip 2 in Advanced Settings.

I used 15 training images and trained 150 repeat 1 epoch.

My used training images are as below (they are at best medium quality)

For RealCartoon-Pixar V8 hopefully I will add Regularization images to this post soon. 

For realism, use our very best real unsplash collected regularization images ⤵️

https://www.patreon.com/posts/massive-4k-woman-87700469

I trained both 768x768 and 1024x1024. 768x768 training works better than 1024x1024. Moreover, generating 1024x1024 works better than 768x768. When fixing faces with ADetailer extension, make the ADetailer extension resolution 768x768 even if you generate images in 1024x1024.

If you don't know how to load configs and use here a tutorial : https://youtu.be/EEV8RPohsbw

Files

Comments

Yoruba

thank you!

DogmaR34

Question, are the reg images/traning images aspect ratio important/considered on this config? Should I crop my input images to be square or leave them in any resolution, thanks!

DogmaR34

Thanks! a further question, im really new at kohya, where do I set the repeats? For reference ive got 191 images

Wktra

no buckets??? 🤯

Arcon Septim

when you used hyper realism, what checkpoint you used for training? fp16 or fp32? also what vae it uses?

AIrtistry Contact

something wrong with the first preset tier_1_quality_slow_22GB.json, took me 2 minutes to finish with an rtx 4090 24 gb from runpod and also when I loaded the config, in the training parameters name it's was the name of the third .json

Steve

Any thoughts on Prodigy over Adafactor? Been reading a lot more stuff from Civitai creators none of the YT "influencers" are pushing it.

Art

apart from the EMA stuff, this is replicable on Kaggle Kohya notebook to train loras?

Patrick Major

I have a question in regards to the "tier_2_speed_11_8GB.json" file. The default LR is 7e-7. I have 24 decent images, no captions and using an img folder name of "75_oshw woman". I also have your reg set for the woman 768x768 being used. When I train, the sample images never ever converge into a likeness of what I'm training on (like normally it would). I think the training rate is way too low, or in your config are you doing something closer to 15,000 repeats? Normally the LR is something between 1e4 and 1e5 so I just want to make sure. I'm running still in kohya as I've only got a 3070 8GB and none of the OneTrainer settings gets me below 10GB.

SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts

actually i trained my model with only 15 images and 150 epoch for this config. so when reg images included it was total 15 * 150 * 2 = 4500 steps. so with 24 images 75 repeating and 1 epoch might not be sufficient. try 150 repeat and it may even become overtrained

Patrick Major

Thank you for that advice, I'll go give it a try. I had noticed the epoch was set to the value 1 in the config file and had thought maybe it was one of those setups where the highest quality was running at 1 epoch and setting the steps high by way of repeats (and then just saving every 500 steps). I'll go give that a try.

SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts

well epoch doesnt mean much since kohya utilized repeating logic. so as you increase repeat you are actually increasing epochs in reality. more repeating better since it uses more reg images. so 8 epoch 40 repeat vs 1 epoch 320 repeat, 320 repeat better

Patrick Major

Thank you again for the clarification. So in your first example to my above question with 15 images and 150 epoch, that would be a good setup for a run without reg images. Are you setting the epoch high in that scenario to make it easier to setup more checkpoint saves? Even with some of my settings off, your base setup is starting to work for me. It's been an obsession of mine for months with some hit and miss results. Your work here is wonderful.

Khoa Vo

What do you use as the prompt for the dataset? Is it something like 150_ohwx_man?

Patrick Major

if you were doing the prompt you are mentioning, in Kohya_ss it would be 150_ohwx man Notice the space after ohwx. The had confused me in the beginning. The images folder in kohya points to something like this: \YourAITrainingDirectory\Subject\Images But the actual images of your topic live one folder deeper: \YourAITrainingDirectory\Subject\Images\150_ohwx man

Patrick Major

I hate to bug you again. I've tried all sorts of ways to get your "tier_2_speed_11_8GB.json" file to work, with the settings recommended. I've already got some good success with other methods, so took a decent dataset where the reality of likeness is around 90% and I used it with the settings. I am training on an Irish woman, the folder path repeats and token being "1_owsh woman" and I'm doing 150 epochs. There are 21 decent quality images. It should way over train with that number. Instead what is happening is during the training, in the sampling (with a prompt of "photo of a owsh woman") I sample every 50 steps and see the system showing me a sea wall, a harbour, then a car and eventually settles onto an older asian woman which has no likeness. I can confirm the folder paths are good, as I can change the LR from 7e-07 to 1e-05 and start to see a decent likeness. I'm using the custom model you recommend in the settings and more importantly the learning rate which you have set to 7e-07. I've tried a variety of image sets that I had prepared, no captions, just the folder tags. To fix it, I just changed the LR to 1e-05 for example. I am raising this more because I think its possible the base example has the LR value set wrong and was going to ask you to look at it as I suspect if you do a training at the exact base settings from the json (with of course modifications for the repeats/epoch combination you choose), you'll find it way undertrains. Unless 7e-7 is for very large datasets? Or do I have to set epochs towards 1000? Something is wrong with the config. It occurs to me now to ask. Maybe I'm using the configs in the wrong location. I'm loading them in kohya_ss in the LoRA tab, are these configs for fine tuning or other? I noticed some learning rate differences between the: tier_2_speed_11_8GB.json and tier_2_lowVRAM_10_2GB.json file, namely in the lowVRAM there are additional training settings that are missing from the tier_2_speed_11_8GB.json such as : "learning_rate_te": 7e-07, "learning_rate_te1": 3e-06, "learning_rate_te2": 0.0, I know the setup us using Adafactor as the optimizer, but the LR Scheduler is setup as "constant" instead of allowing the Adafactor to optimize the LR. Is this intentional? I really liked the results you have in your 6 pages of showing what you can do and am doing my earnest to make your config work. Thanks for any further advice. I'll try a number of settings and report back what I find.

Patrick Major

The "tier_2_lowVRAM_10_2GB.json" is the config I grabbed from this page in relation to the SD 1.5 Koyha best settings training. I am training against SD1.5 as I don't have enough memory to train SDXL locally yet. If you do a quick check on the "tier_2_lowVRAM_10_2GB.json" file you'll notice the LR is "7e-07" and the Unet and TE are 0. I'm not sure if its intentional, but just raising the question as I suspect the values might be set too low. It's not the SDXL config, but the SD1.5 one. Specifically this setting from the json above is what I'm inquiring about: "learning_rate": 7e-07 Am I correct in that I load your above config into the LoRA tab of Koha_ss, or this intended for the fine tuning tab?

Patrick Major

Sorry, I just re-read the title of this wiki. It is for Dreambooth training and not LoRA training. I'm switching over to loading the config in dreambooth tab and trying again, sorry for the trouble. Even in the lowmem dreambooth, I can't get below 10 GB, which means a 17 hour train. I guess runpod will be the option. Did you ever find that you could get the Dreambooth training in your low mem config as low as 8 GB (or slightly lower?)

pravin khetan

there are so many video tutorials on your yourtube channel. Which one should I follow to use your .json file SD 1.5 lora. And how to get best output form automatic1111?

Julius Seizure

yeah it honestly is confusing with the amount of videos there are

Ella Bang

Says that the configs include the models hyper realism v3 to be downloaded from huggingface but it seems to be missing.

Dmitrii

hyperrealism is 2GB size, but my trained output model is 4GB. Why is that? how to make output 2GB as well?

Profile Photo

What number of epochs did you input? 150 epochs ? Because I chose 150 epochs with 15 photos and I have 72 hours of execution which is too much

Profile Photo

Yes I'm using, but my question about where do I input number of epoch, is it epoch parameter or something else ?

min min

amazing work!can you touch us how to make the reg datasets?what's the thought!

Marc

Hi, I would like to know which “tier” I should choose for the Onetrainer configuration. I have a pc 16gb Ram, Nvidia 3060 RTX Ti 8gb, A1111 SD 1.5 checkpoints, I work to obtain an embedding for consistent character face. My second question is: Is there a way to train embedding or lora on a whole body, in order to obtain a consistent character at the level of the whole body? Thank you in advance and also for your kindness and excellent work, which is a great help.

SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts

hello for 8 gb cards you dont have much options. i dont think it can do fine tuning but only a lora perhaps. but i dont have a good lora option either. try tier_2_lowVRAM_10_2GB.json and see if works. what i would suggest you is train a flux model on cloud and use it. it learns body 100% with sufficient images https://youtu.be/-uhL2nW7Ddw https://medium.com/@furkangozukara/trained-myself-with-256-images-on-flux-results-mind-blowing-80e45aad4162

Marc

Excellent. Thank you!