Make Longer LTX-2 Videos + Control & FX in ComfyUI! (Patreon)

Video - https://youtu.be/IwATkI5N6hQ

Overview

Loads of updates for this one, with the most exciting thing being the much longer videos thanks to video chunking!

Extra Features:

VRAM Savings (Over 900 frames!)

Attention Tuner

Audio normalisation

Pose & Canny ControlNet Options

Stage 1 sampler & scheduler select

Extra Upscalers

Image FX

This does mean installing a few extra custom node packs, but I think they're worth it :)

Overall it takes under 60 seconds per video (3090, fp8, 5s), meaning you can make loads of them!

Workflow Information

This almost feels like a video editing program now with the amount of options XD

The extra nodes this time around includes some from the WAN Animate Pre-processing pack, in order to get those nice pose videos, along with RandomInternetPreson's LTX-2 Memory Management nodes and Radiance:

Note that https://github.com/RandomInternetPreson/ComfyUI_LTX-2_VRAM_Memory_Management doesn't install in the "normal" way:

I'm using the comfyui_tensor_parallel_v3 in the workflow.

Note that it does have some limitations at the moment, and so is firmly in the "experimental" group. The main issue is LoRAs:

It doesn't work with ANY LoRAs - this includes the distill, canny & pose ones

As it doesn't work with LoRAs, ONLY the fp8-distilled model works for longer videos

Once enabled, it doesn't actually disable. This means if you switch from the fp8-distilled model to a non-distilled one, the Stage-2 LoRA STILL won't work

One way around is to click both "unload models" and "free model and node cache":

This may change in the future (the GitHub page has this issue raised already), but for now this is the biggest limitation of the node.

Still, I almost always use the fp8-distilled model myself anyway, so if you're like me then it's great :)

The experimental group also contains the audio normalisation and attention modifications:

Normalise Audio is... interesting. It seems like only the two options which aren't set to 1 by default are the only values you can change, and the defaults sucked for me :) Using HIGHER values didn't seem too bad (slightly more "booming" audio), and if you do find any fun values then do feel free to share your discovery with everyone using the comments. Changing the audio also seems to have a small effect on the video produced.

Attention Tuner is also interesting, and again finding "useful" values other than 1 is an on-going mission. It also apparently reduced peak VRAM usage, though at best this seemed be by just 1 GB when I checked using nvidia-smi.

Again, going higher or lower than 1 is an option, and really low values will certainly mess stuff up. Could be useful for testing one block at a time to see what they do...

There are 2 ControlNet options available - Pose & Canny:

To use, you'll need to enable BOTH options - Enable ControlNet and either Pose or Canny.

You will also need to load the relevant LoRA:

For Pose I've gone with the WanAnimatePreProcess nodes:

As a side-note, you can also do a sort of Video-to-Video if you:

Enable ControlNet & Canny Control as usual, but...

DON'T enable the Canny LoRA...

... & instead, Bypass the Canny node in the video loader:

It certainly isn't the best, as you can't change a lot... but for the most part you can change what people in the video say.

(e.g. Trying to change hair colour and background):

(e.g trying to change video style):

Of course, if you use it "properly" (enable canny & use the LoRA) then everything changes A LOT...

Most of the samplers & schedulers work, so I've also added those to the Settings group now too:

If you want to use any of the "classic" schedulers (simple, bets, etc) then remember to enable the "Scheduler - Classic" option.

Image upscaling options are also now available, and for these I've used the "Radiance" pack, though feel free to simply swap any of these nodes for your favourite image upscaling nodes:

There is also an ImageFX group:

Again, treat those nodes as examples, so if you want to replace Radiance Float32 Colour Correct with something else...

... such as the LUT node (if you have and LUT files) ...

... or indeed anything from any node pack you typically use to change images!

If you do have any great Image FX nodes which also work well on videos , then feel free to share your ideas in the comments.

Have fun! :)