Home Creators Posts Import Register Favorites Logout

Downloads

Content

If you are looking for an example model training dataset that is properly prepared, to learn how to prepare and test, this is the dataset!

Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.

Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord

Please also Star, Watch and Fork our Stable Diffusion & Generative AI  GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)

=======

Download zip : Images_Configs_Prompts_v1.zip

Fine Tuned model checkpoints : https://civitai.com/models/911087

CivitAI banned : https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline

LoRA model checkpoints : https://civitai.com/models/918952?modelVersionId=1028569

2 November 2024 Update

Model published on CivitAI as FP8 and FP16 - epoch 170

It has images with metadata : https://civitai.com/models/911087

If you like, review, leave a comment I appreciate

Also we have reddit post, if you like commend upvote I appreciate

1 November 2024 Update

The zip file updated

Now the zip file includes

Dwayne_Johnson_FLUX_Fine_Tuning.json - the config used to train Fine Tuned models

Dwayne_Johnson_FLUX_LoRA - the config used to train LoRA models

Test_Prompts_Realism.txt - Grid test configs new line seperated

Test_Prompts_Realism_Grid_Format.txt - Grid test configs formatted for grid feature of SwarmUI

Grid config - 40 steps, iPDNM sampler, fp16 precision, CFG 1, seed : 180427741

You can download all checkpoints of Fine-Tuning / DreamBooth and LoRA trainings here : https://huggingface.co/MonsterMMORPG/Model_Training_Experiments_As_A_Baseline

Grid Tests

I feel like epoch 170 is the best checkpoint of Fine-Tuning / DreamBooth training

Dwayne_Fine_Tune_Realism_Test_Part1.jpg

Dwayne_Fine_Tune_Realism_Test_Part2.jpg

Dwayne_LoRA_Realism_Test_Part1.jpg

Dwayne_LoRA_Realism_Test_Part2.jpg

LoRA_90_Epoch_vs_LoRA_160_Epoch_vs_Fine_Tuning_170_Epoch.jpg

When LoRA and Fine Tuning / DreamBooth compared, results are close, however when looked at details, it is noticeable that clothing and environment more overfit with LoRA training as expect

Moreover, real difference happens when stylized prompts are used as shown in tutorial video

The grids are not cherry pick so with generating like 10 images you can get amazing images with each prompt

Images zip file : Images_Configs_Prompts_v1.zip

I have been getting asked to publish a proper image training dataset that you can test.

Here a perfect dataset for you to test with Stable Diffusion 1.5 (SD 1.5), SDXL, Stable Diffusion 3, Stable Diffusion 3.5 Large, Medium, FLUX and all other text to image models

Total number of images are 28 - can be perfectly used with our FLUX Batch Size 7 config

When you download the attachments zip file, you will get maximum quality 1024x1024 cropped images

When you pay attention to the images you will notice that it has the following features:

Maximum quality

Amazing lightning

Perfect focus

Perfect sharpness

Variety of clothing

Variety of backgrounds, different places and times

Variety of distances such as head shot, close shot, mid shot and full body shot

So that the model can learn generating every distance

No other person or animal in any of the images

Both smiling and regular expression / emotion

So model can generate both, you need to add others if you need others like angry or shocked, etc.

You can use this dataset to test configurations

I recommend using dwayne johnson as training tokens but you can also use ohwx man as usual if you wish

If you use dwayne johnson you will leverage existing knowledge of the model

Hopefully I will use this dataset to try to fix bleeding problem of the FLUX models when doing multiple-concept / multiple-subject training

This is another reason why I prepared this dataset and hopefully starting testing today on FLUX dev and FLUX de-distilled models

Style Training Dataset

Style training dataset and full workflow and experimentation of style training is shared in below Hugging Face repo

https://huggingface.co/MonsterMMORPG/3D-Cartoon-Style-FLUX

Files

Comments

Tan

Thanks for sharing this. I have been trying to train a person's hands. It is not the face and not particularly a style but can hands be treated as style? What are your thoughts in terms of how many images that would take. I tried making lora but I saw a lot of overfitting.

Tan

Dr, thank you for replying. Yes, I am trying to finetune a Flux Dev model. It will be trained with images of hands and nail art. Here, the prompts will be mostly for close up of hands or fingernails or nail art. Although hands can be small if you consider a full body image, here, I am planning on training it specifically for extreme close ups so that nail artists can use it (starting with a family member). Also, the data set contains photos from just one person and have flexibility to create wide variety of it.

mihail_md_do

thanks fo sharing! one question about emotions. i want my model to be flexible in demonstrating different emotions. should i specify emotions in captions for photos?

Nikola Popovic

so when we use 28 good images for dataset, lets for example myself, different expressions, outfits, background and train locally how many epoch would be needed for best results (i presume you are using batch number of 7 on some cloud machine, I am using 3090 locally so batch would be 1 - if i get correctly)

Pew

Doc, this reminds me of that which I've yet to attempt. Have you, or do you know others, who have trained a Flux character (use case: less ideal quality dataset made better by FLUX), then using the cherry-picked Flux character images for the end-use-case of training an SDXL model for example. Where using the original training images straight to SDXL would have resulted in a poor training outcome, if at all?

Steve

Shouldn't perfect LoRA testing be on characters that the model isn't already trained on?

Tan

I remember reading it in one of your post that captioning matters less. While I'm trying to find the post, I wanted to ask if you used captioning on the Dwyane finetune or Lora.

Tan

Got it, thank you!

Tan

Doc, my first results came in for a specific person's hand. It is pretty good but I am feeling it can be better. I used 532 images for 40 epochs using batch of 7, all images were manually cropped. I used a trigger word along with the class 'hand'. Some images come out perfect but still struggles with a few hand poses. I had also trained Lora which performs better every time (but shows signs of overfitting) but the finetuned version looks much natural and integrated into the image. Question - Can I rename the 40 epoch model and start retraining it again so that I can pick up where I left last time?

mihail_md_do

i made a little research too. i compared finetunes trained with same photos, with same presets (48gb batch 7). one was trained without captions, another with accurate captions. for my dataset captions matter, person looks more similar to real photos.

Luiz xd

I see that the data set is very good... how did you get that quality in the images? with supir x2 x3 x4 .......???

P

Hi, Furkan 👋 How to preprocess the data like i am currently doing only 1024*1024 6-8 images for face portraits, what are some good settings/dataset prepration to produce best possible results, i also use length captions, on ai-toolkit. thanks

P

thanks for the response! so you are saying just use "ohwx man" caption in .txt file instead of the detailed caption, also what about the Lora rank and optimizer and steps? also I would like to ask about the dataset resolution what if we use 2048, and 1024 varied dataset res. size will it impact the loRa?

P

thanks for your response, i tried with only 'ohwx man' in the caption .txt file, optimizer=adamw, steps = 3000, Lora rank 128, learning rate = 0.0001. but it started picking like earbuds, and other accessories from the dataset whereas I wanted to pick only faces. Also, it gives a little bit of plastic-like skin. I have tried the same settings with 4-5 sample datasets from 5 images to 20 images.

Oleh 23

Hello! What is currently the best pipeline for training LoRA models? I’ve heard Flux Gym is really good for this. For dataset creation, is it still best to use SDXL with img2img + ControlNet, or are there better ways now? I’m especially interested in generating high-quality frames for training face and full-body LoRA. Any updated recommendations would be amazing — thanks again!