WAN 2.2 Fun - Control + FLF2V + More! (Patreon)

Video - https://youtu.be/1CP5twTDtxw

Overview

Wan 2.2 Fun Control 14B gives us even more control as it supports things like depth, pose or canny ControlNet videos.

Even better still, from my point of view, is that you no longer have to pick between a Text2Video or Image2Video model for whatever it is you want to do, as this model supports both!

I've got so many models that it's hard to keep everything on my 2TB drive, but seeing as WAN Fun seems to do everything it may be an opportunity to free up some disk space and get rid of the earlier WAN 2.2 models ;)

This workflow has everything in one place, so you only need to enable the particular option(s) you want. Do be aware that if you want to run everything at once, you'll need a decent amount of RAM (128GB+ suggested).

VRAM-wise 24GB remains a decent starting point, but with block swap etc I'm sure it will work on less, just a bit slower!

Example workflow screenshot:

Personally, I only pick one option at a time, but the various choices available by default are:

Text to video

Video to video

Image to video (first, last, others)

Control video to video (depth default)

Control video to video (pose default)

Control video to video + flf to video (depth default)

Control video to video with reference image

Of course you can pop in whatever control video you like, or whatever pre-processor you fancy. I tested depth, pose and canny with others giving sub-optimal results (such as normals, etc)

There are plenty more combinations you can play with, so do let me know of any fun ones I've missed, but I think I've covered the major ones!

Looking at Kijai's 5B examples, swapping in the WAN Fun 5B model loaders should be possible too (as they look very similar), though I've not had the time to test as yet.

Workflow Info

All the model files you can grab from Kijai's Huggingface pages -

https://huggingface.co/Kijai/WanVideo_comfy_fp8_scaled

https://huggingface.co/Kijai/WanVideo_comfy/tree/main

Custom nodes I've tried to keep to a minimum, and you can install any custom nodes using ComfyUI Manager.

Main nodes:

ComfyUI Wan Video Wrapper - https://github.com/kijai/ComfyUI-WanVideoWrapper

KJ Nodes - https://github.com/kijai/ComfyUI-KJNodes

Video Helper Suite - https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite

RGthree - https://github.com/rgthree/rgthree-comfy

For Control Net stuff you can using things like:

Kijai's Depth Anything v2 - https://github.com/kijai/ComfyUI-DepthAnythingV2

And for more options (pose, canny, etc) -

ControlNet aux - https://github.com/Fannovel16/comfyui_controlnet_aux

Have fun! :)