ComfyUI LTX2 + Custom Audio = Perfect Lip Sync & Motion! You Can Make Your Own MV! (Patreon)
Content
Video: https://youtu.be/AqyyLY_ajTQ
In this video, we dive deep into the latest capabilities of LTX2 with ComfyUI, showing you how to fully control AI-generated video using your own custom audio input. Unlike basic image-to-video setups, we’re using video-to-video generation with LTX2 LoRA models and ControlNet—paired with native audio conditioning via the LTX Audio VAE Encode node. This allows for synchronized facial expressions, natural lip movements, and dynamic motion that matches your voice or music track, all generated locally without extra plugins.
This tutorial is perfect for AI creators, digital artists, indie filmmakers, and YouTubers who want to produce high-quality, expressive AI avatars or stylized music videos using open-source tools. Whether you're experimenting with talking head avatars, animated narrations, or full-blown AI music videos, this workflow gives you precise control over timing, motion, and emotional expression—something static image-to-video simply can’t deliver reliably.
Why does this matter? Because LTX2 in ComfyUI now supports true audio-aware video generation, letting you go beyond generic prompts and create content that reacts to your audio. With features like audio latent encoding, frame-count syncing, Stage 1/Stage 2 sampling, and distilled LoRA detailers, you get studio-level polish without cloud costs or proprietary software. If you’ve been waiting for a local, open-source alternative to commercial AI video tools—this is it.
Resources:
--------------------------------------------------------------------
LTX-2 Model Card
https://huggingface.co/Lightricks/LTX-2
ComfyUI-LTXVideo Custom Nodes
https://github.com/Lightricks/ComfyUI-LTXVideo
LTXV2 Models include GGUF
https://huggingface.co/Kijai/LTXV2_comfy
🅛🅣🅧 Gemma 3 Model Loader (follow the Text Encoder from LTX Team)
https://github.com/Lightricks/ComfyUI-LTXVideo?tab=readme-ov-file#required-models
Gemma 3 (For high VRam) https://huggingface.co/google/gemma-3-12b-it-qat-q4_0-unquantized
Gemma-3-12b-it-bnb-4bit (For low VRam) https://huggingface.co/unsloth/gemma-3-12b-it-bnb-4bit
Comfy-Org/ltx-2 - gemma_3_12B_it.safetensors
https://huggingface.co/Comfy-Org/ltx-2/tree/main/split_files/text_encoders
ComfyUI-VoxCPMTTS
https://github.com/1038lab/ComfyUI-VoxCPMTTS
ComfyUI-VoxCPMTTS
https://github.com/1038lab/ComfyUI-VoxCPMTTS
--------------------------------------------------------------------
Have Fun!