Video Guide - https://youtu.be/zwQOe8rSqBM

Overview

OmniVoice TTS is a very fast TTS with voice cloning capabilities - even from just a few seconds of reference audio!

It does seem to work very well, and is a big step up from things like Tortoise TTS from all those years ago :)

In their own words:

OmniVoice is a state-of-the-art massive multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it generates high-quality speech with superior inference speed, supporting voice cloning and voice design.

It has a web interface and a Python API, but of course I've been playing with it in ComfyUI.

I've tested out a couple of options, both of which seem great!

GitHub Link - https://github.com/k2-fsa/OmniVoice

Huggingface Space - https://huggingface.co/spaces/k2-fsa/OmniVoice

OmniVoice Comfy - https://github.com/komikndr/omnivoice_comfy

OmniVoice Comfy - https://github.com/Saganaki22/ComfyUI-OmniVoice-TTS

Workflow Information

Depending on which set of custom nodes you prefer, I've attached a workflow for each. Or you could install both packs!

KomikNDR

These custom nodes don't come with automatic voice transcription, so I went with Nvidia Qwen Canary as the transcription option.

These nodes do require a bit of "manual maintenance" as you'll need to make new directories and download model files yourself. You'll also need to rename the audio tokeniser model when downloading.

NOTE: This node pack also hardcodes package versions in it's requirements.txt file:

transformers==5.3.0

This should be changed to:

transformers>=5.3.0

I had transformers 5.5.0 installed already by default, so for me this would have downgraded that package. To this end, I'd suggest using the git clone option over ComfyUI manager. You can, of course, always upgrade transformers again afterwards so no big deal.

Model links and full instructions are included in the workflow itself:

For the Canary audio transcription I found the bitsandbytes package was missing from the requirements.txt file, so if you don't have that already you'll need to pip install that as well:

Note that you don't have to call the sub-directory "Canary" in models/asr - it can be any name you like.

If you just want to do manual transcription (or you need to correct the auto transcription), then a manual option is available:

The switch in the OmniVoice group can be used to toggle between manual and automatic:

If using voice designer in either node pack, there are only a limited number of words which seem to work:

These can be mixed and matched, such as "american accent, male, very high pitch".

Despite the node default t_shift value of 1, I found that to produce very poor quality results. The defaults in my workflow are therefore different:

Play around with the settings to see what gets the best results for you. Personally, I found the generated audio to often be quite fast, hence the slightly slower speed. Turning postprocess_output off can also help to maintain a more natural rhythm.

Whilst you can save from the preview audio node, you have to remember to do so plus it will save .flac audio files. Comfy also comes with a Save Audio (MP3) node, which you may prefer:

Saganaki22

The Saganaki nodes are slightly different as each task has its own node. I've added bookmarks for ease, and added get/set nodes to tidy things up:

These nodes come with support for Whisper and they automatically download models, so certainly a much easier setup!

There are a number of Whisper models available, but "Turbo" is fast :)

Again, the reference audio inputs have the 3-band EQ attached, but as this custom node pack supports more than 1 reference voice, you may with to clone more groups:

The default node values for these custom nodes seemed a lot better, but again tweaking the settings can produce some better results:

Again, you may like to tweak the speed and postprocess_output values if your generated audio is too fast and breathless.

The LongForm node is almost identical, but also features a chunking option:

Voice Design has it's own node, which I've also attached the Save Audio (MP3) node to for ease. This can be used on any of the audio output connections, so use whichever one you prefer.

Also unique to this pack is the multi-speaker option:

This one is definitely loads of fun, and you can have more than 2 voice if needed.

Summary

Overall, I think both node packs are great, but I do slightly prefer the Saganaki22 option as automatic downloads and multi-speaker are awesome, plus it doesn't have any hardcoded values in the requirements.txt file.

Whichever node pack you use, OmniVoice does need an up-to-date version of the transformers package. I already had 5.5.0 installed anyway, but some older custom nodes may rely on older versions.

If you haven't updated in years or use really old nodes, it is one to watch out for. You may like to clone your current environment if this is the case, so as to not mess anything up, e.g.

conda create --name comfyui_clone --clone comfyui

Whichever pack you go with, I think these are both great options to combine with other workflows which allow custom audio (such as my LTX ones)... or just to play with on their own!

If you enjoy the Canary voice transcription and like being able to see that text and thus modify it, that is an easy copy-pasta from from komi workflow to the saga one. Whisper seems to work just great by itself though, so hey!

Enjoy :)