Capybara - A Unified Model (Patreon)

Video - https://youtu.be/eRutNtJ9MlU

Overview

Capybara - https://huggingface.co/xgen-universe/Capybara

Looking for a model that does:

Text2Image

Text2Video

Image2Image

Image2Video

Image Editing

Video2Video

Welcome to Capybara!

From my testing so far it doesn't seem quite as hot when it comes to photorealism (when compared to other models), but it does have a whole range of styles and is pretty good at oil paintings!

Image generation is fast, though video generation can take a few minutes.

Workflow Information

Model download links are available via the note in the workflow itself.

You won't need many custom nodes, and chances are you'll have all of these install already anyway!

The model loader uses the "full" size model by default, but you can switch to fp8 for the weight_dtype & it uses less than 12GB VRAM to generate the default rodent painting image:

Fp8 mode seems to be roughly the same speed to me, and quality isn't that much different.

You won't always need the vision model loaded, so you can bypass that group when just doing text2image (for example):

The Control Panel gives a quick-and-easy way to enable/disable any groups you want to use before you move on to prompting:

AI Prompt Enhance needs ollama installed, but you can then use any LLM you like :)

If using Text2Video mode, you should increase the empty latent length from 1 (e.g. 81, or 121)

If using any groups which require image input, the image loader is down near those groups:

With the same going for the video loader:

Use the Load Video node to set the width, height and number of frames.

For things such as Image to Video which don't use the empty latent, you can set the number of frames via the HunyuanVideo15ImageToVideo node:

For the Stage 2 & Compare groups, set the GET node to your choice from Stage 1:

Image start with IMG and latents start with LAT. This means you could start with TTI, use image upscale first, the latent upscale and finish with Ultimate Upscale if you like!

A prompting an resolution guide is also included:

Image to Video can be... interesting. Most of the time it helps to describe the image first, then say what happens afterwards. Personally, I've also has more luck there with the linear quadratic scheduler, but ymmv.

Enjoy!