LongCat - WAN Video with Extensions! (Patreon)

Video - https://youtu.be/NMDJSCI_gCM

Overview

LongCat - a WAN Video based model that lets you concatenate lots of scenes together into a single coherent video :) Now you can easily join clips together to get 20,30, 40, etc seconds of coherent video!

Describe each scene with it's own prompt, so you can generate several seconds at a time and then change things as you go along.

Not too bad on VRAM usage, with 480p video showing up as using around 12GB VRAM via nvidia-smi.

Here is what they say -

Key Features

🌟 Unified architecture for multiple tasks: LongCat-Video unifies Text-to-Video, Image-to-Video, and Video-Continuation tasks within a single video generation framework. It natively supports all these tasks with a single model and consistently delivers strong performance across each individual task.

🌟 Long video generation: LongCat-Video is natively pretrained on Video-Continuation tasks, enabling it to produce minutes-long videos without color drifting or quality degradation.

🌟 Efficient inference: LongCat-Video generates $720p$, $30fps$ videos within minutes by employing a coarse-to-fine generation strategy along both the temporal and spatial axes. Block Sparse Attention further enhances efficiency, particularly at high resolutions

🌟 Strong performance with multi-reward RLHF: Powered by multi-reward Group Relative Policy Optimization (GRPO), comprehensive evaluations on both internal and public benchmarks demonstrate that LongCat-Video achieves performance comparable to leading open-source video generation models as well as the latest commercial solutions.

Read more at https://huggingface.co/meituan-longcat/LongCat-Video and LongCat-Video Technical Report.

Workflow Information

This ComfyUI workflow uses Kijai's nodes and I've tested the fp8 model, though there is a larger version available too. For best results use the bf16 model for both LongCat & the text encoder, though you will need a reasonable spec computer (16+GB VRAM, 48+ GB system RAM) .

Model links are available in the workflow:

VRAM usage is recorded in the block swap node group, with results coming from nvidia-smi during rendering:

As it joins lots of smaller video clips together, there is a unified "settings" group:

As it happens, the initial render for T2V or I2V can go as low as 6-steps, but then the further renders seemed to get the typical discolouration issues with that many steps. Their code example uses 16 steps, and that does appear to give better results, though more steps = slower.

Personally I found that using HIGHER resolutions (720p) than the listed set of default bucket aspect rations gave me better results, with 480p videos having a bit of "burn".

There are multiple prompt groups, for example:

There are 4 by default, but you can simply duplicate the groups to have 5,6,7,8, etc different prompts. It is a good idea to change the constant name when duplicating.

To go with the prompt there is also a sampler group:

You should change the video in/out number and the prompt number if you're adding extra groups, as detailed in the note:

Prompting can be a little hit and miss, and it's typically best to be very explicit about exactly what you want to happen in each scene. Describe things in detail, and the model will do it's best to accommodate!

Have fun! :)