Home Creators Posts Import Register Favorites Logout

Downloads

Content

Hello my friends. One of my Patrons recently asked me what the best text-to-speech model is — and I didn't have an answer.

So I did some digging and found that my knowledge was outdated. And as you know me, I couldn't let that sit. I dug deeper and found that no reliably working implementation existed at the time I decided to make a video about it.

So I developed a ComfyUI plugin that works.

All you have to do to run it is update my nodepack to version 1.10.1 via the ComfyUI Manager.

On a restart dependencies get downloaded.

The model is about 10GB in Size. The download starts automatically when you run the workflow for the first time. Same goes for the Whisper Speech to text model.

If you need help installing ComfyUI, I wrote an installer helper, currently available in two different versions.

For all Paying Members

Installer Old Version:

https://www.patreon.com/AIKnowledgeCentral/posts/minimax-h3-and-2-167237656

Installer New Version Preview — currently available for higher tiers only (this will become available to all paying tiers with the next updates):

https://www.patreon.com/AIKnowledgeCentral/posts/installer-v3-and-169981628

I list these so you can see exactly what is under the hood. Nothing hidden, nothing magic. If you ever want to understand why something behaves the way it does, that is where you look.

Used ComfyUI Nodes:

https://github.com/Into-The-Latent/ComfyUI-IntoTheLatent-Utils

https://github.com/pythongosssss/ComfyUI-Custom-Scripts

https://github.com/rgthree/rgthree-comfy

Models used:

https://huggingface.co/BreezeBlue/Breeze-TTS-2

https://huggingface.co/openai/whisper-large-v3

Create something amazing

Chris

Files

Best Local AI Voice Cloning and TTS Model

This open-weights text-to-speech model runs entirely on your own machine, costs nothing, and scores higher than ElevenLabs on the Artificial Analysis leaderboard. No subscription, no character limit, no cloud — just ComfyUI and about 10GB of model files. In this video I walk through the complete BreezeTTS setup in ComfyUI and three ready-made workflows you can drag straight into your canvas. You'll see voice design, where a plain text prompt plus a written instruction like "a woman panicked, voice trembling with fear, fast shaky pace" produces a fully directed performance, including breaths, throat clears and laughs. Then voice cloning from a 10 to 30 second sample, with Whisper speech-to-text built into the workflow so the transcript of your reference audio is generated and fed in automatically — no typing it out by hand unless you want to. Finally the third workflow combines both, letting you clone a voice and then redirect its tone, pacing and emotion, which is easily the most powerful of the three. I also cover installing the missing custom nodes, the version requirement for the ComfyUI-to-the-Latent pack, where the model downloads from, and realistic generation times so you know what to expect on your own hardware. 📩 Stay up to date with Local AI & Open Source AI https://intothelatent.com/newsletter ❤️ Support the channel & get exclusive workflows, guides and early access https://patreon.com/intothelatent" 📅 Book a personal 2-hour 1-on-1 AI coaching session (paid offer) where we'll solve your specific workflow, ComfyUI, Stable Diffusion, or local AI questions together. https://koalendar.com/e/1hr-1-on-1-stable-diffusion 🏠 My Website https://intothelatent.com Movie clip from Taken (2008), directed by Pierre Morel. © EuropaCorp. All rights reserved. Used for commentary/illustration purposes. Image from https://villains.fandom.com/wiki/Shadowheart https://breezeblue.ai/ https://huggingface.co/BreezeBlue/Breeze-TTS-2 https://huggingface.co/openai/whisper-large-v3 Chapters 00:00 Intro 00:45 Installing the Missing Custom Nodes 01:52 The Simplest Workflow You'll Ever Build 03:59 Sneaking In Laughs and Throat Clears 04:40 Cloning a Voice from One Sample 07:05 Clone the Voice, Change the Energy and Tone 09:05 Pushing the Model with Grunts, Gasps and Non-Verbal Sounds

Comments

No comments found for this post.