Ovi - The "Veo3-like" for ComfyUI (Patreon)

Video - https://youtu.be/mkXo5c_N9qw

Overview

Ovi is a "Veo3-like" you can run at home on your own PC which creates video and audio together!

🎬 Video+Audio Generation: Generate synchronised video and audio content simultaneously

📝 Flexible Input: Supports text-only or text+image conditioning

⏱️ 5-second Videos: Generates 5-second videos at 24 FPS, area of 720×720, at various aspect ratios (9:16, 16:9, 1:1, etc)

Depending on your choice of file and block swap, you can run this with 8GB VRAM or more:

Prompting

Ovi is based on WAN for video generation with MMaudio for audio.

There are three sections to the positive prompt:

Your typical scene description

The audio to generate, enclosed within <S> and <E> tags

Audio style description (optional), enclosed within <AUDCAP> and <ENDAUDCAP> tags

Example:

A super-high quality film clip of an amazingly pretty female space alien wearing a white and gold outfit. In the background is a futuristic holo-monitor screen. The alien is saying, <S>"Why yes! I do actually exist INSIDE your computer."<E>. <AUDCAP>Clear, British female voice, well-spoken, very feminine, high quality recording.<ENDAUDCAP>

There are also TWO prompt boxes:

The first prompt node contains the full positive prompt and the negative prompt for the (WAN) video generation.

The second prompt node functions as a negative prompt only and applies to the audio generation.

Unfortunately this won't work 100% of the time, and sometimes scene text may get mixed up and end up being spoken. Putting speech in 'quotes' (or "quotes") can help to mitigate this, but still isn't 100%.

I found dpm++sde/beta often generated better audio, better text and followed the prompt better, but with the overall video sharpness not always being quite as good as the default unipc.

Different values for Skip Layer Guidance can generate better/audio video... but it really is a mixed bag and I couldn't find any consistently better options :/

Video Generation Options

Select which options to use:

Note that you must enable at least option 1 (Text-2-video) or option 2 (Image-2-Video) for video generation. If neither of these are selected then a video will not be generated.

It is suggested that you keep the Experimental Options enabled as Skip Layer Guidance can have a reasonably positive impact on your generations.

The "experimental args" are not connected by default, but feel free to connect and test as you like!

Video Resolution

Note that it is possible to select an INCOMPATIBLE resolution using the WanVideo Empty Embeds (or image resize) node. You will get an error in this case.

Attachment Info

The workflow is set to a low resolution 480p generation by default which will gives results quickly and allow you to test the prompting. Once you're more familiar with how it all works then it's time to crank up the resolution to the full 720p!

There are TWO sets of model files available:

If you have less than 12GB VRAM, you'll likely be better of sticking with the FP8 option.

Note that whichever set of files you opt to download, the MMAudio VAE files are only available via the Option 2 link.

You will also need some custom nodes installed:

These are pretty common, so chances are you'll have them already :)