Home Creators Posts Import Register Favorites Logout

Downloads

Content

Video - https://youtu.be/7AhQcdqnwfs

Flux is a great model with loads of potential, but when it comes to generating images with almost any art style it can struggle a bit. You may also wish to add your own face, a pet or indeed anything you have some images of!

Woodblock art style without a lora...

Woodblock art style WITH a lora :)

Ai-Toolkit Quick Install with Conda

git clone https://github.com/ostris/ai-toolkit.git

cd ai-toolkit

git submodule update --init --recursive

conda create -y --name my-ai-toolkit python=3.11

conda activate my-ai-toolkit

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

pip install -r requirements.txt

For alternative pytorch installs, see https://pytorch.org/

Changes for Microsoft Windows

Users of Microsoft Windows could use this command instead:

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Also, to install git if you don't have it :

conda install git

Datasets

4 to 40 images should be fine to start with

Use the optional Florence Captioner to create matching text files

Links

Online - https://fal.ai/models/fal-ai/flux-lora-general-training

Local - https://github.com/ostris/ai-toolkit

Anaconda - https://www.anaconda.com/

Miniconda - https://docs.anaconda.com/miniconda/

Files

Comments

HealingPaint

I've installed all missing modules in Comfy, and added some SAVE IMAGE ones, but cannot clear this error..."When loading the graph, the following node types were not found: Save Image" what am I missing? What is the exact addon I need to install for this to load the Florence Captioner workflow?

HealingPaint

Is there another way to get my images captioned quickly? I'm stuck here before I can run the training I have otherwise ready to go...

Jeroen Van Harten

I had exactly the same issue, even after installing YANC in custom nodes, and even though the included save images node was missing inputs. However, after reloading workflow, it finally did work for me. How? No idea.

Ronny Khalil

does resolution/aspect ratios of image data set matter at all?? from your video it looks like not. it’s usually the most tedious part of training for me. o happy day if i don’t have to 512x512 my lora fodder anymore

HealingPaint

Now it works to output images, and shows the TEXT being figured out in the CMD window, but no TXT files are appearing in the folder...I've tried a few output folders for the text as well...ideas?

NerdyRodent

you'll need to set 3 paths: 1. The directory with all your images 2. The full path to where you want to save the text files (e.g. /home/myuser/comfy/output/my-lora) 3. The path relative to your ComfyUI output directory to save the image files (e.g. my-lora)

Jens Huebner

After creating a image folder, when I try to run it, i get: Running 1 job Error running job: No module named 'ipadapter.ip_adapter' Any Ideas how I can fix that?

HealingPaint

I have all 3 paths set. the images complete fine but the TXT files never appear in any path I set. It acts like it will work, shows the descriptions in the CMD window, but they don't appear on disk. Ideas? I'm setting the same path for both images and txt files.

NerdyRodent

Try searching for the file 0.txt in case you've got a typo in your directory path and it's not saving where you think

Jens Huebner

Thank you, that did the trick. But now I got stuck with downloading the diffusion_pytorch_model-00001-of-00003.safetensors ... it just didn't do anything, even after waiting one hour it was still 0% So I am loading them manually now from https://huggingface.co/black-forest-labs/FLUX.1-dev/tree/main/transformer I think I have to put all five files from there into this folder? E:\ai-toolkit\checkpoints\flux-dev\transformer Can someome confirm this? Sorry if this is a stupid question. I am totally new to all this.

HealingPaint

Sadly no text files are found. It does show the images (in the workflow itself) after running it. It shows TEXT in CMD and in the SHOW TEXT module in the workflow itself which reveals that the CLIP is working to describe the images, but no text files are being created... it sucks to be this close and not able to do this. :-( (I'm just using the workflow defaults and changing the path only BTW)

NerdyRodent

Very strange! As you're getting the images output to the directory you specify it won't be a permissions issue. Everything still points to a typo in your text output path. Maybe try making the directory first, then copy & paste the path in?

HealingPaint

The paths are both copy-pasted, identical. tried again, careful with spaces, etc...same issue. No idea. I need to find a different way to caption these images somehow.

Runebinder

Hi, set everything up (on Windows 11), run the command to start training the LoRA and get this error: "OSError: black-forest-labs/FLUX.1-dev does not appear to have a file named config.json". I've deleted the .yaml file and recopied from examples folder and only changed the path to where the sample set it in case I messed something up the first time but same error. I've created the .env file with my Hugging Face token but not joy. Not very good with Python, any ideas?

HealingPaint

I do see this in the command window as well and the text module in comfy is bordered in red. Failed to validate prompt for output 259: * (prompt): - Value 0 smaller than min of 2: filename_number_padding * Save Text File 259: - Value 0 smaller than min of 2: filename_number_padding Output will be ignored

HealingPaint

Changing it to 2 fixed it, but it renames the files not to match the images which are 1.png, 2.png, etc...not the text files are 001.txt, 101.txt, etc ?

HealingPaint

At least I can train now when I rename the TEXT files to match, but I don't know why the workflow is forcing me to use a filename number padding of 2 or higher...can that be fixed?

HealingPaint

Their page says, "12/15/2023 WAS-NS is not under active development." - it's okay, I'll just live with it. Thanks.

Randall Barriga

Hello I have error when executing Florence2Run, The operator 'aten::isin.Tensor_Tensor_out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS. do you know how to fix that?

Jens Huebner

Thanks again. That worked. But now I got stucked while downloading shards. It just isn't do anything. I waited several hours yesterday but it still was 0%. Any ideas what I can do? Running 1 process Loading Flux model Loading transformer Quantizing transformer Loading vae Loading t5 You set `add_prefix_space`. The tokenizer needs to be converted from the slow tokenizers Downloading shards: 0%| 0/2 [00:00?, ?it/s] model-00001-of-00002.safetensors: 0%| 0.00/4.99G [00:00?, ?B/s]

NerdyRodent

Sounds like you're probably on an Apple Mac? One for their github issues by the sounds of things!

NerdyRodent

It could be something to do with the country you're in, so perhaps a VPN would aid the internet connection? Essentially, you need some way to download the files from huggingface - either by letting it do it automatically, via a web browser, or any other downloader

Randall Barriga

yes you are right, I'm on mac, but I don't understand what you try explain me, where do I need to go exactly?

Albert Bukoski

after I use install nodes, tensorops fails to install, either from manager or comman line, even with a 'pip install requirments"

The World According To Gar (edited)

Comment edits

2026-09-14 18:00 Ok. I just joined. I did the install, apparently successfully. I wanted to run the workflow to create the training data. But, lo and behold, no start.bat for comfyui. How do i initiate comfyui. I have a few other installations in other directories. But, is there one in the ai-toolkit directory that this installation created?
2024-08-21 05:35

NerdyRodent

I don't see tensorops listed in the requirements file for either Florence2 or LLM Party, so not sure what's going on there. Check for errors as ComfyUI starts up, or as an alternative you could start up a new, clean comfyui environment which should be free of any issues.

Lúcio Nunes

Good day, Nerdy and folks. I'm getting this error when trying to train my lora: python run.py train_lora_flux_24gb_luciov0.yaml Running 1 job Error running job: cannot import name 'apply_rope' from 'diffusers.models.attention_processor' (/home/comfyui/ai-toolkit/ai-toolkit-env/lib/python3.12/site-packages/diffusers/models/attention_processor.py) Any ideas how to get this resolved?

Lúcio Nunes

Worked like a charm, little gnawing creature! LORA training is in progress. Thanks a bunch!

Jens Huebner

Sorry to be a pain in the ass. I tried several proxy servers and VPN but same result. I am trying almost a week now. It's frustrating but I am still not willing to give up. Is there maybe a workaround to solve this problem?

NerdyRodent

A few workarounds off the top of my head: 1. If you're using Microsoft Windows, instead boot a Linux Live CD to do the download 2. Use git clone to download the repo instead of using a web browser 3. Use huggingface-cli to download the repo instead of using a web browser 4. Use a public internet service (such as a local library, or internet cafe) and transfer the files back to your system on a USB drive

Guillaume C.

Hello When I launch the florence project, I have a couple of issues with unknown nodes Warning: Missing Node TypesWhen loading the graph, the following node types were not found: DownloadAndLoad Florence2ModelFlorenc2Run> Save ImageListCounter //Inspire LoadImageListFromDir //Inspire

Guillaume C.

I already have the manager. I updated again, and it seems better now. But I always have an issue with the "save_image". (stay red) And in the terminal I ve this error message To see the GUI go to: http://127.0.0.1:8188 [ERROR] An error occurred while retrieving information for the 'Florence2ModelLoader' node. Traceback (most recent call last): File "m:\pinokio\api\comfy.git\app\server.py", line 468, in get_object_info out[x] = node_info(x) File "m:\pinokio\api\comfy.git\app\server.py", line 436, in node_info info['input'] = obj_class.INPUT_TYPES() File "M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-Florence2\nodes.py", line 142, in INPUT_TYPES "model": ([item.name for item in Path(folder_paths.models_dir, "LLM").iterdir() if item.is_dir()],) , File "M:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-Florence2\nodes.py", line 142, in "model": ([item.name for item in Path(folder_paths.models_dir, "LLM").iterdir() if item.is_dir()],) , File "m:\pinokio\bin\miniconda\lib\pathlib.py", line 1017, in iterdir for name in self._accessor.listdir(self): FileNotFoundError: [WinError 3] Le chemin d’accès spécifié est introuvable: 'M:\\pinokio\\api\\comfy.gi t\\app\\models\\LLM' [Inspire Pack] IPAdapterPlus is not installed. Error. No naistyles.csv found. Put your naistyles.csv in the custom_nodes/ComfyUI_NAI-mod/CSV directory of ComfyUI. Then press "Refresh". Your current root directory is: M:\pinokio\api\comfy.git\app Error. No naistyles.csv found. Put your naistyles.csv in the custom_nodes/ComfyUI_NAI-mod/CSV directory of ComfyUI. Then press "Refresh". Your current root directory is: M:\pinokio\api\comfy.git\app

NerdyRodent

Excellent! ComfyUI Manager is how I install my nodes and it's not let me down yet! For your error, does the specified path exist? Likely it's a pinokio specific issue, so it may be worth going with a standard install instead to test.

omgwtfh4x

This was an incredible video + guide. This was my first dip into the AI pool and everything worked like a charm. I created a Patreon account specifically to follow your page. Thanks for the incredible write up.

Ayumaou (edited)

Comment edits

2026-09-14 18:00 When loading the captioner, it says Warning: Missing Node Types When loading the graph, the following node types were not found: > Save Image No selected item Nodes that have failed to load will show as red on the graph. But when I want to install the missing ones with the manager it cant find anything. I used your git link from below but it is still not working.
2024-08-31 04:26

Ayatanne

I have the same problem and I installed that from your link, but the save image problem is still there.

Ayumaou (edited)

Comment edits

2026-09-14 18:00 Can you share your Woodblock art style lora, I like it a lot
2024-08-31 05:00

Guillaume C.

I had the same issue, but no solution for the moment. I deleted the nodes, it saved some txt files, but the result seems good only for 3-4 files, after the caption doesn't correspond to the image

Guillaume C.

The captions don't correspond to all images. The number of txt is not associated to the good image :(

NerdyRodent

It's likely you've got conflicting nodes installed, so it's using the other one you have instead of the actual one in the workflow. Try uninstalling or disabling any conflicting nodes.

Ayatanne

maybe your images start at 1 but the text at 0, you should make sure it both starts at the same number

ThoughtFission

No matter what I tried, I just couldn't get this to work.

NerdyRodent

Start with a clean install of Comfy so you know you've got a working install, then add the extra nodes from there :)

Peter Rossa

so I m learning to do that training, I was updating nodes via manager and now ComfyUI doesnt work on Opera (it just opens a blank white tab with no nodes nothing) but it does work in firefox, weird xD, (and I cant get that fancy save image custom node to work I did cloned it from here: https://github.com/ALatentPlace/ComfyUI_yanc) but it still shows up red in the captioning workflow

Peter Rossa

sorry for spamming xD ok I have failed on creating .env file, Im doing something wrong I dont know where to put the .env file and how to exacly activate my READ token, after executing training I got this messege: D:\FreshSDXL\AI_toolkit\ai-toolkit>python run.py config/train_lora_flux_24gb_example2.yaml Traceback (most recent call last): File "D:\FreshSDXL\AI_toolkit\ai-toolkit\run.py", line 5, in from dotenv import load_dotenv ModuleNotFoundError: No module named 'dotenv'

NerdyRodent

You save your .env file in the root directory and in "HF_TOKEN=your_key_here" replace 'your_key_here' with your actual key (from after you login to huggingface). The text file called .env is unrelated to the python module. From your example output there, it looks like you've forgotten to activate your ai-toolkit environment first - hence it can't find anything

Peter Rossa

thx, I think I will do it all again xD I got another error: D:\FreshSDXL\AI_toolkit__OLD\ai-toolkit>python run.py config/train_lora_flux_24gb_example2.yaml Traceback (most recent call last): File "D:\FreshSDXL\AI_toolkit__OLD\ai-toolkit\run.py", line 22, in from toolkit.job import get_job File "D:\FreshSDXL\AI_toolkit__OLD\ai-toolkit\toolkit\job.py", line 3, in from toolkit.config import get_config File "D:\FreshSDXL\AI_toolkit__OLD\ai-toolkit\toolkit\config.py", line 5, in import oyaml as yaml ModuleNotFoundError: No module named 'oyaml'

edenaion (edited)

Comment edits

2026-09-14 18:00 When I load the Florence_Captioner, the Save Image node is still red even after installing all missing nodes. Any ideas?
2024-09-06 16:59