MoCha - Wan Video Character Replacement! (Patreon)

Video - https://youtu.be/vWTnlEspiOg

MoCha Overview

Another week, another model and this time it's MoCha. A bit like WAN Animate, only without the need for a pose/depth/canny video. All it needs is a single image (ideally closely matching the first frame of the video) and that's it!

You can change the face, the clothing, the background... whatever you decide to try :)

The downside? This one is pretty VRAM heavy as even with block swap at 40 it'll chew through 12+GB VRAM for an 81 frame 480p video generation, so 16GB VRAM+ is recommended!

Seems to handle longer videos (81+ frames) pretty well in most cases, though I didn't test past 10s. The longer the video, the more VRAM you'll need :|

Tested with the context node also, but that didn't seem to work too well... do let me know if you get any better success using that!

Workflow Information

The workflow features various sections including:

Qwen Image Edit built-in for easy "first frame" editing

Prompt-based masking

Video Super Resolution (optional)

Full workflow screenshot example

This workflow has Qwen Image Edit "built in" so you'll need those models too in order to use that option. It's a cut-down version of the one from https://www.patreon.com/posts/qwen-image-edit-140305178 which uses only the masked image.

Torch Compile can be bypassed or removed if you're using MS Windows and haven't installed triton - it's purely for a small performance boost.

Masking is prompt based this time, which makes things a bit easier!

Note that while the masking works great for Qwen, MoCha can sometimes ignore the masked area and change other stuff!

There are various options available, but note that you do need to have at least 1 reference image (either via Load Ref1 image, or via Qwen Image Edit):

Ref2 is for an optional face image to help maintain the face similarity... but as to how well it works... that's another matter. For the most part it did seem OK with a single image anyway, but some faces would change a little.

As for custom nodes, it's much the same as Wan Animate, and FlashVSR actually has a few options available.

The "NoVSR" version is the same workflow, simply without the VSR option as... well... I wasn't super impressed with the version I tested as it seemed to over-sharpen a lot of things! I can also be a bit of a hassle installing the custom node, so just go with the NoVSR option to skip it altogether.

The one I tested was FlashVSR Ultra Fast - as shown below:

However do note that installing via ComfyUI Manager may not be the best idea for this particular node due to the fixed versions in it's "requirements.txt" file.

There's nothing in there that I didn't have installed already, so in my case a simple git clone of the repo was all that was required.

If you do need to "pip install -r requirements.txt" then you may like to remove the first few lines from the file first. The fixed versions for pytorch and numpy could mess things up, and simply removing the files saves changing anything you have in your environment already. Yes, I like to have the latest versions installed rather than specific things from a random custom node ;)

You will then also need to make a "FlashVSR" directory in ComfyUI/models and download the various files into it.

The other FlashVSR options I haven't tested, but their "requirements.txt" files don't seem to have fixed versions and may or may not be better!