Lightricks - LTX 2.3 in Comfy UI (Patreon)

Video - https://youtu.be/eFdfNcVIpD0

Overview

LTX 2.3 is here and its a bigger & better version! Audio and video quality is improved, and it should follow your prompts better too!

LTX-2.3 is a DiT-based audio-video foundation model designed to generate synchronised video and audio within a single model. It brings together the core building blocks of modern video generation, with open weights and a focus on practical, local execution.

As I like to test things to the extreme, I've set a rather difficult default prompt because who doesn't like shifting timelines? ;)

You can do a whole variety of things including:

Text to Video

Image to Video

First / Last Frame

Custom Audio

Upscaling

Video FX

Generation time can take anything from seconds to minutes depending on things such as frames, resolution, sampler and your own hardware.

Works best with a decent amount of both VRAM (16GB+) and RAM (64GB+), but ComfyUI has also recently made a bunch of optimisations that should work great in the latest versions. See https://github.com/Comfy-Org/ComfyUI/discussions/12699 for more information & be sure to update if you haven't already!

Workflow Information

The two main workflows attached differ only in the model loader groups.

The Non "KJFiles" version uses the standard Lightricks checkpoint file option:

The "KJFiles" workflow features the use of individual files instead:

There is also a variation of the "KJFiles" file without Radiance nodes (only Comfy base & KJNodes) with the NoRad suffix.

So, whichever model files you prefer to use, there is a ready-made workflow for you :)

Note the the official Lightricks fp8 distill model hasn't yet been uploaded (at the time of writing), so for the moment only the "KJFiles" version has a distill fp8 model.

You will need to install a few custom nodes, as detailed in the workflow note:

No more TTP Toolset nodes in this one, or RandomInternetPreson VRAM saver nodes as they have both been replaced with nodes from KJ Nodes.

The VRAM saver (on by default, because why not?):

The new First/Last image group:

Note that you can also use more than 2 images via KJ Nodes:

You will need to add more image loaders and set the index for each of your files, so simply duplicate the image loaders and image resize nodes as required. Probably a good idea to use some more GET/SET nodes for particular indexes too if you're going that route!

LTX2.3 seems OK with the old IC LoRAs, though I image we'll likely see some new ones soon. To this end, I've also updated the Pose Detection group to use the new "SDPose" nodes:

If you prefer the previous pose detection group, that can be copy/pasted from the previous LTX2 workflow - https://www.patreon.com/posts/make-longer-ltx-149020375

You can find the links for all the models in the workflow notes:

Just click to download whichever ones you fancy! Remember to check the repo itself for various file version as there are quite a few options.

One notable change is the inclusion of a 1.5x upscaler as well as a 2x one:

The default uses 2x, but the 1.5x may give you some better results.

The various settings now also have their own colours and groups, making it easier to see the groupings:

You can quickly enable or disable various options using the fast group mute/bypass switches:

Note that some options will override other options. For example:

Empty image latent has the lowest priority, then control net, then initial image then first/last image. Thus, if you opt to use first/last image then that latent will override the controlnet latent.

You can still use the first image option as provided in the first image group (Bypass Image If Using ControlNet?) , just not a last image:

For the most part you won't need to ever even look at the "dragons" area:

but feel free to explore and change if you're feeling adventurous!

By default, the two AI Prompt Enhancer nodes don't have the Get IMG node attached. If you do want the LLM to look at either IMG1 or IMG2, then don't forget to attach the GET node(s) as shown below:

You can click the arrow to change IMG selection, e.g.

Have fun! :)