Home
Creators
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register Favorites Keys Review DMs Logout
Community
Telegram ThePornDude
Home Creators Posts Import Register Favorites Logout

AI Enhanced Flux (Patreon)

Published:
2024-08-07 20:30:02
Imported:
2026-06
Tags:

Downloads

Content

Flux Updates!

Things are moving along nicely in the Flux.1 world and soon we'll have things like controlnets, loras! But, for now at least, we've got image-to-image and LLM assistants. These ComfyUI workflows are similar to the previous ones, but updated to use the standard "KSampler" and "Load Checkpoint" nodes. The checkpoint for this is different - https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors - though the reduction in quality isn't too bad.

I should really look into using the image input for LLM party, and it looks like there are so many more things you could do as well!

Links

Examples: https://comfyanonymous.github.io/ComfyUI_examples/flux/

Florence: https://github.com/kijai/ComfyUI-Florence2

LLM Party: https://github.com/heshengtao/comfyui_LLM_party

Ollama: https://github.com/ollama/ollama

Workflows

As always, the workflows used in the video can be found attached to this post!

Files

Comments

Oliver Baddeley

Got it working, but you are using the ays sampler... I'm not sure how to install that... I'm Googling, but thought I'd ask like a real pesky rat.

vrai honnête

Sorry to bother, ss the "LLm party" workflow not included?

Oliver Baddeley

Any chance you gonna open up a Test Rodents feed? Maybe a day or two ahead of your YouTube release so that we can help you iron out the little issues like these before the whole world sees your glorious rodentness?

Oliver Baddeley
>>145958584

It’s the first json in the list. Patreon does appear to take a second to get use to where the links are. I had the same issue first day here too.

lichthorizont

i got flashattention2 error in the second json-file ... would you show me how to insatll it please? Error occurred when executing DownloadAndLoadFlorence2Model: FlashAttention2 has been toggled on, but it cannot be used due to the following error: the package flash_attn seems to be not installed. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2.

p tjie
>>145959835

i got ollama installed and started, but still get error regarding to "flash attention 2" would someone help me to install "flash attention 2" please?

troyau
>>145959835

Had the same error. Just change the attention to sdpa, seems to work fine.

Stephen Purvis

Hi NerdyR I got the same error regarding flash Attention 2 and have been trying for hours to install it to no avail, can anyone help with that. however I did switch to sdpa and it worked then. FlashAttention2 has been toggled on, but it cannot be used due to the following error: the package flash_attn seems to be not installed. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2.

What to watch high (edited)

Comment edits

2026-09-14 18:00 "Error occurred when executing CheckpointLoaderSimple: ERROR: Could not detect model type of: P:\AI\ComfyUI\models\checkpoints\FLUX1\flux1-dev-fp8.safetensors" Comfy is up to date. Any ideas?
2024-08-08 16:00

What to watch high

"P:\AI\ComfyUI\nodes.py:1446: RuntimeWarning: invalid value encountered in cast img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8)) Prompt executed in 1681.19 seconds" Well that was a fun 30 minutes.

NerdyRodent
>>145977274

Yup, "pip install -U flash-attn --no-build-isolation" should get you the latest version if you need it!

NerdyRodent
>>145985552

You'll need to do both of these steps before you can run the workflow: 1. Update ComfyUI - E.g. ComfyUI: 2489[8115d8](2024-08-07), Manager: V2.48.5 2. Download the checkpoint - https://huggingface.co/Comfy-Org/flux1-dev/blob/main/flux1-dev-fp8.safetensors - into your ComfyUI models/checkpoints directory. You can make sure you've got the right file via the SHA256 and model size - SHA256: 8e91b68084b53a7fc44ed2a3756d821e355ac1a7b6fe29be760c1db532f3d88a Size of remote file: 17.2 GB. On windows, that would be something like "certutil -hashfile C:\file\path\to\flux1-dev-fp8.safetensors SHA256"

troyau

For anyone with limited vram and don't want to run a llm locally give openai a try with their new GPT-4o-mini model. It's $0.15 per million input tokens and $0.60 per million output tokens. Around 300 requests, using it in comfyui, over the last day or so has cost me 3 cents.

NerdyRodent
>>146025582

ComfyUI-ppm is the one you want for ays+ without using the custom sampler. See the notes in the workflow for more info!

lichthorizont

if i install flash attention 2, i got this error: C:\StabilityMatrix\Packages\ComfyUI\venv\Scripts>pip install -U flash-attn --no-build-isolation Collecting flash-attn Downloading flash_attn-2.6.3.tar.gz (2.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 6.1 MB/s eta 0:00:00 Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [12 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 14, in File "C:\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\setuptools\__init__.py", line 8, in import _distutils_hack.override # noqa: F401 File "C:\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\_distutils_hack\override.py", line 1, in __import__('_distutils_hack').do_override() File "C:\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\_distutils_hack\__init__.py", line 70, in do_override ensure_local_distutils() File "C:\StabilityMatrix\Packages\ComfyUI\venv\lib\site-packages\_distutils_hack\__init__.py", line 57, in ensure_local_distutils assert '_distutils' in core.__file__, core.__file__ AssertionError: C:\StabilityMatrix\Packages\ComfyUI\venv\Scripts\python310.zip\distutils\core.pyc [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. any Idea how to solve it? Thank you in Advacne

Lúcio Nunes
>>145959835

I was getting the very same black image when using sdpa. I installed Flash Attention using the command "pip install flash-attn" and now everything works like a charm.

I am Groot

Is there a way to make a workflow with only Flux and the LLM party? I don't have a need for the img2img part and im also having the flash attention 2 error. I get errors when updating "pip install -U flash-attn --no-build-isolation".

NerdyRodent
>>146157337

For any node you don't need, left-click on it and then press the "delete" key to remove it from the workflow. Or, to remove an area of nodes all at once, hold down "ctrl" along with the left mouse button to create a selection box! Remove the IMG2IMG section and you've basically got the same as setting the switches to 1,2,1/3. I try to organise things into groups too, so another option is to simply use bypass.

Nibmeister

I can't find the nodes CR Latent and Get Image Size. Where to get?

Nibmeister
>>146175242

Never mind. I ran out of disk space and reinstalled ComfyUI on another disk and all is well.

Remco Bos

Finally got it to work after fighting cuda, flash-attention-2 compilation (windows). it's awesome!

J
>>146189886

Same issue here. Been trying to install it but it can't build the wheel. Tried installing it using comfyui manager's "install pip packages," which seemed more promising, but still failed.

Mark
>>146189886

Which version of CUDA be you running my friend? I have managed to install it but its not being recognised as installed. Do you have to have a certain version of pytorch & CUDA?

Remco Bos
>>146189886

In short, I compiled Flash Attention 2 from source: pip install flash-attn --no-build-isolation I'm using Cuda 12.4 and pytorch 2.4.0+cu124 For compilation on Windows 11, you need Visual Studio (I used 2022), ninja, packaging. Compilation took more then an hour. I used ChatGPT along the way and that was super helpful: https://chatgpt.com/share/a1c074d9-49af-439f-a0fb-201e5185fec6 (it also contains some unrelated questions, I will remove the chat log in the future)

Kryptonic
>>146189886

you can also change the attention option in the node to one of the other options instead of flash attention 2, still seems to work fine. Had that problem awhile back when I first tried out Florence.

Kryptonic
>>146061115

try a different attention option in the Florence node, like eager or sdpa, worked for me when i first started using Florence

Brendan Commons

Can't seem to get the ays+ scheduler. Tried messing with other KSamplers and some gave AYS options but not a single 'Latent' node output and not the ays+ scheduler. Is the scheduler super important to the workflow? EDIT: figured id try git clone again since the ppm folder seemed awfully empty and sure enough i didnt clone in admin cmd first try. But this still didn't resolve the issue. Failed to validate prompt for output 9: * KSampler 145: - Value not in list: scheduler: 'ays+' not in ['normal', 'karras', 'exponential', 'sgm_uniform', 'simple', 'ddim_uniform', 'beta']

ALISON REIS
>>146189886

I did all of this and I've been at it for 2 days trying to resolve it. It installs but still gives an error: Error occurred when executing DownloadAndLoadFlorence2Model: No package metadata was found for flash_attn File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ I'm very frustrated, I signed this Patreon just because I really liked the workflow and what was shown, but this is a point that should be addressed in the video considering so many people have this problem.

TigerFox57

Dearest Nerdy. In the workflow for the florence 2 stuff, the width and height ouputs of the get image size node are connected to the wrong inputs of the maths expressions, so a portrait image comes out landscape, and vice versa.

NerdyRodent
>>146378633

If neither of the 3 options for Florence are working, check the output when ComfyUI is starting it as there could be errors shown there.

TigerFox57

I'm very confused. I didn't mess with the workflow until after I had processed an image that was in portrait and it came out in landscape. So I looked at the maths nodes stuff, and swapped them round, and now my images seem to be coming out with the correct aspect ratios. I assume that the numbers in the nodes called "Math Expression" that are connected to the node called "Get Image Size" should be the results of the calculation in the box. However, my input image has dimensions 640 Width and 1173 Height, at least according to Windows file manager, but in the Maths boxes, the one connected to the width output says 1376, and the one connected to the height output says 752. It's bizarre! Am I going mad?

TigerFox57
>>146407781

Yes, it is installed. I went into the manager and disabled it, restarted comfy, and that node then showed an error. So I'm baffled now.

NerdyRodent
>>146407781

There you go, that's what it is then. Be sure to use the one specified in the note next to it - GetImageSize = stability-ComfyUI-nodes

TigerFox57
>>146407781

Aha, your ratlike cunning has triumphed! I swapped the YOLO node for one from Derfuu, and that produces the correct(ish) numbers in the maths boxes, and all is right with the world again. Although I still don't understand how the floor function is producing the numbers it does, but that a problem for another day. Many thanks for your patience.

ON A LES CROOO

Hello ! when i start Florence img2img v1 i have this error: Error occurred when executing DownloadAndLoadFlorence2Model: Error no file named pytorch_model.bin, model.safetensors, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\ComfyUI\models\LLM\Florence-2-large. File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Florence2\nodes.py", line 90, in loadmodel model = AutoModelForCausalLM.from_pretrained(model_path, attn_implementation=attention, device_map=device, torch_dtype=dtype,trust_remote_code=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\models\auto\auto_factory.py", line 559, in from_pretrained return model_class.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\SergioLucas\Desktop\ComfyUI_windows_portable\python_embeded\Lib\site-packages\transformers\modeling_utils.py", line 3460, in from_pretrained raise EnvironmentError

ON A LES CROOO
>>146574515

ok.. i try to reinstall florence2 and it seems fixed ! Flashattention 2 does not work, even when downloaded : cd hopper python setup.py install I still managed to launch with eager or sdpa . But I can't figure out what changes between the 3 attentions? ( sdpa , eager , flash_attention_2) When I launch, I can get a description of my text, I then try to put the text in cliptextencodeflux to get a similar image: the image I have in output is completely black, and comfyui doesn't seem to indicate an error. Do you know how to solve this problem? Another thing: Why do we use the Florence-2base model and not Florence-2large? Also, how to install ays+ in the scheduler..? THANKS

NerdyRodent
>>146574515

There's barely any difference between the 3 attentions, so just use any :) You're also free to use any Florence model of your choice! You'll need ComfyUI PPM for ays as a non-custom sampler. See the notes in the workflow for more information.

ON A LES CROOO
>>146574515

Ok and last question for me : Why when i try to put a number greater than 1 in the batch_size, there is only one image that comes out?

NerdyRodent
>>146574515

Did you remember to set the switch to empty latent when changing the empty latent batch size? I guess you must have a very high VRAM GPU!

ight
>>146189886

same here :( im on windows, and i cant get the 2 workflow with "Florence" to work, quite disappointed... I get "FlashAttention2 has been toggled on, but it cannot be used due to the following error: the package flash_attn seems to be not installed. Please refer to the documentation of https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-2 to install Flash Attention 2." error , so then I installed Visual Studio C++ and tried to run "pip install flash-attn --no-build-isolation" but i get another error "c:\Users\xxx\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\include\torch/csrc/python_headers.h(12): fatal error C1083: Cannot open include file: 'Python.h': No such file or directory error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2" and now I am stuck and dont know what to do :(

ight
>>146189886

if i change "attention" to "eager", i get a black image output.. (but no errors)

ight
>>146574515

how did you fix the black image ? I have same issue as you

NerdyRodent
>>146189886

All 3 options are pretty much the same, so if flash attention isn't installed then you can use either of the other two. Florence doesn't produce any images here - only text captions - so if you're getting black images, check for other errors in your setup. So long as you see the text output from Florence, it's doing it's thing!

delete_me

hi save file node is undefined. Where do I get that. thanks

Trevor Baxter

@NerdyRodent I was playing with these and notice that most have a scheduler of ays+. Easy to change away from and run them, but if you think it is a good one I'd like to try it. Where the heck do you get it though? I too have the Florence issues. I tend to use the preview version of VS2022 though and the CUDA stuff seems sensitive to that, so I just pick a different attention model instead.

NerdyRodent
>>150427215

The info for ays+ is in the workflow note right next to it. I've pasted it below for reference also: "denoise = 1 for empty latent denoise = <1 (~0.85) for image keep CFG = 1.0 ays+ = https://github.com/pamparamm/ComfyUI-ppm Select empty or img latent" Windows is indeed a bit of a headache when it comes to anything AI! One reason I switched to Linux was for the easy life where everything just works :)