Home Creators Posts Import Register Favorites Logout

Content

Qwen Image LoRA results from more than 35 Stage 1 trainings with SECourses Musubi Tuner; the published pre-made configs train on GPUs with as low as 6 GB.

Images were generated in SwarmUI with the 8-steps Lightning LoRA plus the trained LoRA in 8 steps and a 2x Latent Upscale to 2656x2656, from a weak 28 image dataset. Stage 2 research will hopefully improve quality even more; the 1-click installer runs on Windows, RunPod and Massed Compute.

Qwen Image LoRA trainings Stage 1 results and pre-made configs published - As low as training with 6 GB GPUs - Stage 2 research will hopefully improve quality even more - Images generated with 8-steps lightning LoRA + SECourses Musubi Tuner trained LoRA in 8 steps + 2x Latent Upscale

1-click to install SECourses Musubi Tuner app and pre-made training configs shared here : https://www.patreon.com/posts/137551634

  • Hopefully a full video tutorial will be made after Stage 2 R&D trainings completed

  • Example training made on the hardest training which is training a person and it works really good. Therefore, it shall work even much better on style training, item training, product training, character training and such

  • Stage 1 took more than 35 unique R&D Qwen LoRA training

  • 1-Click installer currently fully supporting Windows (local), RunPod (Linux & Cloud) and Massed Compute (Linux & recommend Cloud) training for literally every GPU like RTX 3000, 4000, 5000 series or H100, B200, L40, etc

  • 28 images weak dataset is used for this training

  • More angles having dataset would perform definitely better

  • Moreover, i will make a research for a better activation token as well rather than ohwx

  • After Stage 2, I am expecting hopefully much better results

  • As a caption, i recommend to use only ohwx nothing else, not even class token

  • Image prompts randomly generated with Gemini 2.5 in Google AI Studio for free

  • Download app and configs : https://www.patreon.com/posts/137551634

How to Generate Images

  • In the zip file of this post : https://www.patreon.com/posts/114517862

  • We have Amazing_SwarmUI_Presets_v21.json made for SwarmUI

  • Import it and i am using Qwen Image 8 Steps Ultra Fast to generate images and then apply Upscale Images 2X to make them 4x resolution (1328x1328 to 2656x2656)

  • Of course in addition to preset don't forget to select your trained LoRA - I used LoRA strength / scale = 1

  • This tutorial shows it : https://youtu.be/3BFDcO2Ysu4

Related posts

All posts in this topic: LoRA and Fine-Tuning Training: Kohya, Musubi Tuner, OneTrainer, AI Toolkit · New to SECourses? Start Here: Best and Most Popular SECourses Posts

Files

Comments

Brad Loflin

Amazing work. Thank you.

SECourses

you are welcome thanks for comment

Markus

Thanks a lot. But honestly I am very disappointed. These images scream "generated with ai", they so far from photorealistic photos

cool1

Maybe the skin texture is a bit smooth compared to flux1 dev/krea? Though it's a higher res model so in theory it shouldn't. Though maybe it does different art styles better than flux1 dev, and it is supposed to be better at text/prompt following. I'll try the latest config for 24 GB cards. Hopefully it will be a lot faster than the >= 15 or 18 mins ones I tested previously. Ideally the same image prompts could be compared with the flux models so we can compare how real looking both are and ideally the expected time per training step could be shown in the different GPU presets (though it would likely need to be tested on those first).

cool1

Thanks for adding the new presets/configs and the new testing. In the 24 GB preset, the bucketing is disabled. If we have multiple aspect ratio training images I assume we should set it to enabled, though I assume it could also increase VRAM/RAM usage and so possibly go over the 24 GB preset VRAM usage. Hopefully using bucketing won't slow it down. I can check the debug info you said you added I think. Also in the 24 GB preset the network alpha is set to 128 (as well as network rank). Though the help says "1.0 for Qwen Image on the Network Alpha field". So I assume the 128 should be set to 1 if I'm training a Qwen image lora? edit: mine's a lot faster with your config than my previous tests, thanks. It's at 19.15 secs /it as of step 10 (so faster than the multiple minutes it was taking on my try. (edit: and sample images output a lot quicker than previous tests of this). I did also move the main base model to a faster drive but I assume that's not what's speeding up each step. Slower than Flux1 dev training but I assume it's expected when it's 1328x1328 and a bigger base model/better prompt understanding.

Ken

it will be very helpful if you could add configs for wan 2.2 also

SECourses

no dont change anything in preset unless you know like changing alpha. those info were written based on musubi tuner. i will remove them. and yes you need to enable bucket but it shouldnt increase vram unless your total resolution be bigger than 1328x1328

cool1

Okay thanks. I'll change that back to net alpha 128 when I next restart it. I did think that (like Kohya ss I think used to do, if max bucket size was high enough (double training maxres), that if training res was 1024 that if you allowed buckets and a max bucket size of 2048, that if it found any that were 2:1 aspect ratio and at least 1024 size then kohya ss would I think set the lowest side length to 1024 and the highest side length to 1024*the image aspect ratio (to the nearest bucket size) so eg. if it was a 2:1 image (eg. width twice the height) it would train those particular images at about at 2048x1024 I thought. Though I don't know if this musubi training is doing something like that as like I said, I don't see things like "max bucket size". And though I've given it a folder to output logs in I'm not seeing any logs there yet, even though it's part way through training. Though I haven't specified a logging tool. I'm not sure if I'm supposed to do that for it to actually log.

cool1

Also for the output samples, it doesn't seem to be reloading the prompt file for each set of output samples. It would help if it did reload it each time so that you could reduce the number of image prompts it uses for certain outputs or copy alternative ones to use from the original prompt file (as long as they're ones it's cached). eg. on the flux1 dev presets that cache everything (I think the dreambooth ones. Other ones (flux1 dev Loras I think) it doesn't need to cache them all and you can change the prompt.txt file as much as you want while it's training) once it's cached all the prompts in the prompt.txt file you can create a shorter prompt.txt file for it to use and then only copy from the full prompt.txt file if you want to add a few more prompts or change some. If it's not going to reload the prompt txt file on each set of output samples it's going to output every one of the original ones each time, taking an unnecessary long time - many minutes if you put a lot in (eg. you might only want to have 1 to 3 prompts in there until it has gone through enough learning steps to add more).

SECourses

sadly i couldnt get better realism yet. the model itself is really stylization based. lets see after stage 2 if it gets improved and i plan to train with 256 images

cool1

Now when I try running it after re-starting training (after I set net alpha to 128 instead of 1 and reduced the number of prompts in the prompt.txt because for some reason it isn't re-loading the prompt.txt at each time it starts outputting the samples), after re-starting it it now gives the error: "ValueError: Unsupported logging capability: (none). Choose between ['all', 'aim', 'tensorboard', 'wandb', 'comet_ml', 'mlflow', 'clearml', 'dvclive']" when "(none)" is in the logging section (which was the default). Yet it worked with that before. I don't know what each of those different log types do or the benefit of each. Though I've now changed "(none)" - what was the default to tensorboard and it's started training now. Though I don't see why it's needed when it wasn't before. Also I don't see how I can see that logging info (it's not outputting a log file yet). Maybe it's some command prompt thing you need to run to see it I think but it doesn't show that info in the GUI. edit: also it would help if the filenames of the output samples were named in a sequential order so that the new ones always appeared after the earlier ones when you sort by filename. eg. if you enable "sample at first" the first samples are appearing in the folder further down than the more new files (when they would be better if they didn't).

SECourses

ye i dont know exact details you need to ask kohya

cool1

The "keep last N Checkpoints" doesn't seem to be working properly for Loras. I had that set to 3 but it left 8 loras there (saved every 400 steps), and "Keep last N state files" was set to 3 too, but it kept all 8 state folders (saved every 400 steps).

SECourses

Keep Last N Checkpoints (Alt) Alternative checkpoint cleanup for step-based saves. Keep only last N checkpoint files. 0=keep all. Only active if 'Save Every N Steps' is used

cool1

Thanks. I'll set that value then. Maybe the "Keep Last N Checkpoints" could show that it's for epoch based saved then if it its, eg. say "Keep Last N Checkpoints (for epoch based saves)". I was using both epoch based saves (which it saved 2 of before I stopped training) and saving every 400 steps (which it saved 8 of).

cool1

In the tab for "Image Captioning with Qwen2.5-VL" it seems like: 1) It doesn't seem to generate captions that are as long as you ask it to, and 2) Unlike Joycaption beta 1 and COGVLM2 there's no custom prompt option to tell the captioner how to create the caption (where you can give it extra information like the person's name or tell it what things to caption or what the location is etc.). In the "Image Captioning with Qwen2.5-VL" option if you ask it to caption cityscape-8027996.jpg from pixabay (this was just used as a test/example), with use fp8 precision = false, max image size (for processing) =1280 max new tokens=2048, do sample =true the caption it generates is only 69 to 84 words in the few tests I did. -- The same image with Joycaption beta 1 caption type: descriptive caption length: 260 max new tokens 512 generated 214 words -- The same image with COGVLM2 (v4 of the gradio app) max new tokens: 512 enable sampling: false generated 135 words So the qwen 2.5 vl for some reason generates a lot shorter captions than you ask it to (ask it for 2048 tokens max, giving it a detailed image, and it generates only up to about 84 words, when for the same image joycaption beta 1 (with the "up to 260 words" option) generates 214 words. For 2) allowing a custom prompt for captioning, perplexity.ai said "Qwen2.5-VL itself supports prompts but that functionality might not be fully exposed in that interface.". So if qwen 2.5 VL allows custom prompts for captioning, could that functionality please be added to the GUI?

SECourses

there is literally custom prompt option Custom Prompt

cool1

Thanks. Sorry I didn't notice that. I've seen it now. I've just tried it with "max new tokens" of 1024 and a custom prompt of "create a long caption for this image of at least 300 words." to test it and it's created a caption of 408 words, which is longer than before where it only created up to about 84 words despite setting a high max new tokens (even of 2048). So it must have been whatever was used as the default prompt (before I added mine) that caused to create the short prompts before I think. So that seems to have fixed it thanks. edit: I'm assuming the musubi tuner trainer for qwen will handle captions up to around 400 words (or more) - when doing the training, since the qwen model handles more than that according to AIs. Though 400 words would be around 600 tokens but I assume he's increased the maximum amount of tokens allowed in image captions for training above about 512 tokens that was the limit in kohya ss trainer (for flux1 dev) because qwen can handle more.

cool1

In a discussion on github in the musubi tuner discussion, kohya-ss said to someone (who was only using 3 training images originally) "Also, dim=128 seems too high, and personally, I always set alpha to 1 to simplify learning rate estimation." So he's saying he always sets (network) alpha to 1 to simplify learning rate estimation. According to perplexity.ai: "Because [network] alpha controls the strength of the LoRA weight contribution during forward passes and effectively scales the updates, setting alpha to 1 makes the training more conservative, less likely to overshoot, and can avoid weights zeroing out or collapsing. Consequently, training with a lower alpha (like 1) will take longer because changes per step are smaller, but can lead to fuller, more stable, and potentially more accurate details."

SECourses

i think dont go over 512 tokens . but i dont know what is max

Eclipse Beast - TF

Can I run this locally? How long will it take to train a character on 10 photos with 24 GB VRAM? I don’t know whether to rent a server or if it could finish overnight

doug m

I'm getting RuntimeError: Latent caching failed with return code 1 -- and I created cache_dir to match the toml

SECourses

please look screenshots. give parent folder path and click generate dataset button

SECourses

yes you can run. it is not very long try preset. it depends on steps count

cool1

In your testing have you tried any tests with net rank >128 if it supports that? kohya ss GUI supported over 128 net rank. Though your previous GUI used to say under the net rank "[RECOMMENDED] LoRa rank/dimension. 16 for Qwen Image. Higher = more capacity bug larger files. Range: 8-128". It no longer shows a range under it now. I assume the text was removed because qwen image should likely have net rank >16. But if it supports>128 net rank I assume that could give better quality for the Lora, even though i twould probably increase VRAM/RAM and training time to increase it a bit above 128. I assume the "recommended" in it meant those were only the recommended settings, and if so maybe it's worth trying over 128.

SECourses

over 128 should work but i didnt test. the reason is that when you go over 128 the benefit is so little. i think above 128 doing fine tuning / dreambooth more logical

cool1

Are the results of the newer Lora training tests coming soon? Is that still being tested or the results checked? Is dreambooth training possible with it? Someone said on on github that he'd run Qwen fine tuning with Musubi trainer (though he originally got out of memory errors). So maybe it could be added to a GUI. Though I don't know if it needs to be done at lower res for <=24 GB VRAM PCs. Though there's also the Hunyuan 2.1 that might be good/better for Loras in future if it's higher res and if it can run and train with the quantization.

SECourses

lora done i will update the post. i am working on dreambooth / fine tuning.

cool1

Thanks.

cool1

For the dreambooth option that's being worked on in the merged main branch, I think it was said works as low as 5 GB VRAM. But doesn't that slow it down a lot? Will there be the option to have it run at the higher VRAM but without going over the amount in our GPUs (eg. say an option for about 20 or 22 GB VRAM use for 24 GB GPUS, whatever would be best). Would it also be possible to have the app so that if you stop a training (either Lora or Dreambooth) that the step and epoch counts will continue from where they left off when resuming from a state file? eg. currently what happens is, if you stop a lora training after eg. 1 epoch or 800 steps, that if you tell it to resume from the last saved state (say from saved epoch 1's state), then it starts the step counts again at 1, and it thinks it still needs to go to from step 1 to the same total steps as before (which it doesn't) and gives a low % complete after restarting. If I stopped it after step 800's state was saved and then resumed it with that state, it would be better I think if the first step it shows after resuming is step 801, and so the % complete it shows would be about the same as it was on the state you're resuming from.

SECourses

yes it doesnt keep step count you have to recalculate. ofc there will be configs for every gpu same as lora hopefully

cool1

Thanks. Since the state's folder name has the number of steps in it I think as part of the file name name when saving every x steps) and maybe there could be a .json in the folder with a field with the saved number of steps. I thought it could just get the last saved step count from there and at the start of resuming it could set a variable like current_step_nr=last_saved_step_count_from_saved_state + 1. Though maybe it affects other things. Musubi recently said to someone "Saving and restoring the number of steps and epochs is somewhat cumbersome due to the limitations of accelerate library used in this project. We would like to address this in the future.". So maybe he'll change it some time in future. It seems surprising though since it probably resumes okay apart from those step counts and epoch counts.

SECourses

yes as long as fully resuming it is working right now. just need to reduce target step count