Home
Creators
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register Favorites Keys Review DMs Logout
Community
Telegram ThePornDude
Home Creators Posts Import Register Favorites Logout

Flux.1 - Speed Boost + More ControlNets & LoRAs! (Patreon)

Published:
2024-09-01 12:51:44
Edited:
2024-09-01 17:45:29
Imported:
2026-06
Tags:

Downloads

Content

video - https://youtu.be/JqEi82Q9Qfc

Hyper-SD have released some Flux.1 loras for you to reduce the number of steps required in Dev down to just 8! :)

If that isn't enough, Shakker & InstantX have partnered to give us all these goodies:

Controlnet Canny

Controlnet Depth

Controlnet Union

Controlnet Union Pro

LoRA Add Details

LoRA Blend Realistic

They all work with ComfyUI core, so no extra nodes to install (other than image pre-processors to easily create inputs for their controlnets)

(example workflow screenshot)

Update!

Update! Apparently the union modes are like this:

canny = openpose

tile = depth

depth = hed/pidi/scribble

blur = canny/lineart/anime_lineart/mlsd

pose = normal

grayscale = segment

low quality = tile

Makes sense :)

Model File Download links

https://huggingface.co/ByteDance/Hyper-SD

https://huggingface.co/Shakker-Labs

https://huggingface.co/InstantX

Attachment info

1. Flux_Dev_TXT2IMG_LLMParty_HyperSD_v2.json = Flux Dev Text 2 Image with LLM Party & Hyper-SD

2. Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_v3.json = Same as above, but with a lora loader stack to more easily use multiple loras

3. Flux_Dev_TXT2IMG_HyperSD_ControlNet_v0.json = Flux Dev Text 2 Image with Hyper-SD & Control Net

4. Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_NoClear_v3.json = Same as 2, but doesn't unload the LLM

Notes

Use ComfyUI manager to install any missing nodes - https://github.com/ltdrdata/ComfyUI-Manager

Use "Update All" in ComfyUI manager to keep up-to-date

Files

Comments

Andro.Meta

One of my favorite workflows in SDXL is taking the union ControlNet and using IP-Adapters style transfer to effectively restyle or reimagine a base image with a new image. I'm excited to plug these technologies together in Flux!

KING LIFER

Yeah.... this doesnt work.

Richard Gates

Hey Nerdy for the Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_v3.json workflow, any idea why the LLM Party section would create a prompt very quickly the first time and then take several minutes to do the next one? I have 32GB of system RAM and 16GB of VRAM. I thought one of your previous workflows with the LLM party was faster, but I could be mistaken.

NerdyRodent
>>148206318

I've added an option "clear model" node for lower VRAM systems. You can change the settings in that node so that "is_enable" = false. That way the ollama model won't unload each time.

Richard Gates
>>148206318

I tried that and for some reason I get this with it disabled. Error occurred when executing CR Text Concatenate: can only concatenate str (not "NoneType") to str File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 317, in execute output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 192, in get_output_data return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 169, in _map_node_over_list process_inputs(input_dict, i) File "D:\AI\pinokio\api\comfy.git\app\execution.py", line 158, in process_inputs results.append(getattr(obj, func)(**inputs)) File "D:\AI\pinokio\api\comfy.git\app\custom_nodes\ComfyUI_Comfyroll_CustomNodes\nodes\nodes_utils_text.py", line 222, in concat_text return (text1 + separator + text2, )

NerdyRodent
>>148206318

I've deleted the node in this version - Flux_Dev_TXT2IMG_LLMParty_HyperSD_MultiLora_NoClear_v3.json

vrai honnête

I find the LLM party addition which has become a staple in your workflows really eats up unecessary ressources while not giving, really, that much of a benefit(in terms of ressource cost) could you think about returning to or maybe making something a little more simplified? Of course it is up to you...I mean, who am I too...really no but who am I....(who am I?)....Thank you for your time!

edenaion (edited)

Comment edits

2026-09-14 18:00 LLM-Party node broke my ComfyUI install due to something with torch, It was a hassle to figure out how to restore it, please continue to create workflows without in the future if possible!
2024-09-04 12:59

NerdyRodent
>>148477243

LLM party doesn't have any fixed version numbers in it's requirements.txt, so it should play well with everything else (I've got over 30 custom nodes installed XD). If you do find you've got other custom nodes impacting your install, just run another environment. (I tend to clone them first when I see custom nodes with hard version numbers in)

NerdyRodent
>>148466061

LLM party only uses the VRAM while it's loaded, so with OLLAMA_KEEP_ALIVE=0 it essentially uses 0 extra VRAM. It's also easily removable if you don't want any LLM enhacements :)

John Doe

Do you recon it would be possible to use xlab's ipadapter and these controlnets in combination with one another?

NetDude
>>148477243

whoa. you make new environments just for the extensions? i mean i guess it IS a party. :) that's a good idea but I always assumed it wouldnt work.

NerdyRodent
>>148477243

I clone environments in quite a few cases, but especially if I'm upgrading or testing custom nodes (especially ones with strict requirements.txt files). It's much easier to just create a new clone should anything go a bit weird and start downgrading packages, etc :)

Christopher Manzione

Hey Nerdy Rodent, Is there a workflow here for Flux that would work for Video to Video?

Christopher Manzione

Also I have the following error on this workflow "The size of tensor a (3072) must match the size of tensor b (38) at non-singleton dimension 3" Any idea why this is happening?