Home Creators Posts Import Register Favorites Logout

Downloads

Content

Even more Flux.1 updates - workflow attached as usual :)

If you've not updated ComfyUI in the last 16 hours, then you won't yet have the new ControlNet support, so be sure to update now :)

Loads of other updates top the workflows:

Optimised node placement so that bypassing IMG2IMG is quicker

Added More notes for all the various options

Added ControlNet - needs CFG 4! Switch back to ~2.1 when not using ControlNet

Added Lora

Added device choice nodes from ComfyUI Extra Models so you can pick devices for the VAE & CLIP. Running CLIP in CPU mode is the same as with the Pixart loaders now, so you can save a little VRAM for very little performance loss. Forcing the VAE to GPU makes the VAE operations a bit faster too

Added optional "magic" negative added with bypass switch

Added optional text box. Handy for forcing things like in the example workflow screenshot. There the denoise is set to 1, so it's only using the controlnet for the image shape - the rest of the description comes from Florence. Adding "Professional, cinematic photo. " as prefix text is a quick and easy way to change certain aspects of the Florence desdcription.

Changed default for the Download Florence Model to eager, which may be better supported on things like Microsoft Windows. I can't really see much difference between any of the choices, so just use whatever you prefer.

Update

Added a TXT2IMG version for those who never want to use images :)

 

Files

Comments

Scott Ownbey

Trying this out, very cool! is there a faster workflow for this? \

NerdyRodent

I'm not seeing much of a speed change from the controlnet myself? With ControlNet: FP8: 16/16 [00:11<00:00, 1.38it/s] FP8/CLIP FP16: 16/16 [00:11<00:00, 1.34it/s] Without ControlNet: FP8: 16/16 [00:12<00:00, 1.32it/s] FP8/CLIP FP16: 16/16 [00:11<00:00, 1.35it/s]

kc

Super helpful! Have not tried or installed Olama yet. Do you reco simply using the Windows EXE from their git? Or is it simpler whereas I just download a model to a folder?

SouthbayJay

I'm having issues with the LLM PARTY nodes.. Import failed for them, that happened last week also and I gave up trying to fix it but now we have another workflow using it so I think I really need help getting it fixed. Anyone else have issues with it? My comfy is all up to date as of an hour ago so that's not an issue...

Richard Gates

I'm seeing that if I disable the IMG2IMG group, it stops because ControlNet doesn't load since it is dependent on that image, and if you disable both img2img and controlnet then the workflow fails because you lose the prompt since it comes out of controlnet. Unless I'm missing something.

Nibmeister

To my surprise, I got the workflow to work on my poor, underpowered 3070TI with 8 GB of VRAM. Now to convert it to the NF4 model, if that's possible.

KING LIFER

wow this took really long to render

kc

got this working btw... brilliant setup. Ollama was surprisingly easy to get going... been beat up by all the python/env/etc past two years.

TigerFox57

Hi. May I ask you if you are able to use ollama locally, and if so how much system RAM do you have. I have the workflow running on my laptop, but I have to use Openai's gpt-4o-mini via their API, as my local ollama install stalls the workflow. The laptop has 16 GB system RAM and 16 GB VRAM.

Nibmeister

The full Flux dev model takes 85s per iteration on my 3070TI but the fp8 model gets that down to about 6s/it.

Richard Gates

No luck here yet. I think the workflow will need to be modified for the bypass to work since it has dependencies from other nodes.

Asyouare

You have a laptop with a 16gb vram card? Are you sure you can't just upgrade your ram? Must be possible. Sometimes these things are a little fussier than they need to be but beats spending over one or another couple of thousand. Right?

TigerFox57

I have looked into this, and they claim that 16 GB is the maximum system RAM available. It's a Medion Erazer Beast X30. However, there are hints that it might be possible I might just give it a go, if the RAM isn't just soldered in.

NerdyRodent

There are no conflicting nodes listen in ComfyUI manager, so it shouldn't be that. Some node packages could still potentially conflict though. The import error should give an indication of the issues in your Comfy environment.

NerdyRodent

Yup! You're missing the switches ;) If you're bypassing IMG2IMG & ControlNet, you'll need to remember to set the latent switch to empty and the prompt switch to user input. I've added a txt2img workflow for those who don't want to use images

NerdyRodent

Not 100% sure if the loras and controlnets work with NF4, but maybe? There are some new controlnet & sampler nodes to test as well. The joys of new things!

gloobster

Ran into some serious performance issues whenever I was using the LLM Party section of this workflow on my Win11 box (RTX4060 16GB VRAM, i71400K with 192GB RAM). Generation speed without the LLM enabled was 2.4s/it, and it jumped to ~87.8 s/it with it enabled. It was basically an issue with the LLM and Comfy fighting for/starving each other of VRAM. Solved the issue by doing two things: First: Created a user environment variable named OLLAMA_KEEP_ALIVE with value of 0. This causes OLLAMA to unload its model immediately after returning the response to the workflow. 2nd: Added a node PurgeVRAM from the LayerUtility custom nodes to the end of the workflow. Attached it to the output of the VAE decode in parallel with the Save Image node in your workflow. This force-unloads the checkpoint after the image has been created. After these changes are completed, I'm at 2.43s/it when flux is working, and an overall total image prompt execution time of 65.62 seconds. So I'm paying a penalty of about 10s/image by force unloading the models, but that is way better than the 15+ minutes I had before!

Lord Banzai

Thanks NerdyRodent, looks good! Lots of options. ComfyUI-LLM_Party keeps getting better and better.

Michael TODD Seals

I didn't explicitly enable it, but that is the default behavior for me. If I exceed the total VRAM (16GB), shared VRAM kicks in and performance becomes abysmal. I suppose this is a Good Thing(tm), because sometimes you just peak over max VRAM and then some gets freed up and things go faster.

James

I can see wanting multiple gens with the same ollama description, maybe appending some extra text to it, so I'd make the checkpoint dump at the end optional. Thanks for pointing the setting and node out; I was looking for a way to forcibly dump VRAM since occasionally my 10GB 3080 gets in the same mode where it only runs 87s/it... otherwise, typically I can get a flux schnell image in about 50sec. I'm really hoping nVidia increases their VRAM size options soon. I had a 4060 in my cart for prime day, but after some quick checks realized it would be a processing step-down to gain the 6GB VRAM; so I'm waiting.

Lúcio Nunes

Guys, I'm trying to use the workflow Flux_Dev_IMG2IMG_Florence_Llama_Lora_Ctrl_v10.json, but I'm getting some red circles in the nodes "pythongosssss/ComfyUI-C...", more specifically a red circle in the letter "a". How can I get this resolved?

Lúcio Nunes

I found this warning message here on node management: 【pythongosssss/ComfyUI-Custom-Scripts】Conflicted Nodes (1) ShowText|pysssss [ComfyUI-Universal-Styler]

NerdyRodent

Unfortunately if you have something installed which conflicts with pythongosssss/ComfyUI-Custom-Scripts then you'll need to disable it to use the workflow.

Lúcio Nunes

I tried disabling it, but still I got the error. I'll reinstall everything from scratch. ^^

Nibmeister

Using the Flux Compact fp16 model (combines clip+vae) which loads and runs faster and can use Flux LoRAs.

Lúcio Nunes

Most likely, my dear little rodent. I got everything up and running again after some few hours of pure joy reinstalling, downloading, configuring... I'll be more careful now to not break it again. =F

KING LIFER

Anyone else this goes really show?

Rhellis

Is it possible to tweak this to use just a Clip text encode for positive and negative without the LLM addin?, been playing and dissecting this but to no luck.

Michael Childress

Anyone getting a segmentation fault after installing Comfyui_LLM_Party?

Michael Childress

Im getting a segmentation fault after installing the llm party node. Havent figured out a work around yet.

Herbert Kosch

Which .json to download for the beethofen picture ? The img2img?