How to Install VibeVoice for Free AI Text-to-Speech and Voice Cloning

TL;DR

VibeVoice can be installed locally through ComfyUI for free, offline AI text-to-speech and voice cloning. The open-source Microsoft system supports up to four distinct speakers, outputs longer than 90 minutes, and voice references as short as four seconds in the demonstrations. It also adds context-aware emotion and handles multiple languages, so read on for installation options, workflow details, and tested capabilities.

Transcript

Sam, we need to make AI great again, just like we did with America. Mr. President, AI development is already progressing rapidly with proper safeguards in place. Safeguards? We don't need safeguards holding back American innovation. You should be moving faster. Good day, mate. You all set for the project kickoff this evening? Almost there. Just sor... Read More

Key Insights

• VibeVoice is a free and open-source voice cloning and text-to-speech system from Microsoft that can be downloaded and run offline. Local operation allows unlimited generations without relying exclusively on an online service, according to the tutorial.
• VibeVoice supports outputs longer than 90 minutes and as many as four distinct speakers. These capabilities make it useful for extended formats such as podcast episodes, audiobooks, presentations, and scripted conversations involving multiple recurring voices.
• Voice cloning works with short reference recordings in the demonstrated examples. The tutorial uses clips lasting 22 seconds, 14 seconds, 9 seconds, 8 seconds, and even 4 seconds, then generates new dialogue that retains recognizable vocal characteristics.
• Context-aware expression changes delivery according to the transcript. In one monologue, the generated voice moves from happiness and excitement to disappointment and sadness, then finishes with anger, demonstrating emotional variation without separate performance recordings.
• Multi-speaker generation preserves differences between reference voices. A three-speaker test retains a British male accent, a chipmunk-like vocal quality, and the pronounced expressiveness of a fictional character while assigning each transcript line to its designated speaker.
• Multilingual speech works across the tested Japanese, Spanish, German, Chinese, Korean, and English material. The presenter judged the Japanese and Spanish examples natural, while explicitly noting that he could not personally verify whether the German pronunciation was correct.
• Mixed-language generation can switch languages within one speaker's passage. A test combining English, Chinese, and Korean uses a Japanese reference voice, and the resulting English retains a Japanese accent while the other languages are also presented as accurate.
• VibeVoice can be used online or installed locally through ComfyUI. The tutorial includes separate sections for online use, ComfyUI installation, and operating VibeVoice inside ComfyUI, providing alternatives for experimentation and offline generation.

Questions & Answers

Q: How do you install VibeVoice for free?

The tutorial installs VibeVoice locally through its ComfyUI integration, with separate sections covering installation and speech generation inside the workflow. Because VibeVoice is free and open source, the presenter says the local setup can run offline for unlimited generations.

Q: What is VibeVoice and what can it generate?

VibeVoice is a free, open-source voice cloning and text-to-speech system from Microsoft. It converts transcripts into speech, clones voices from reference recordings, and supports outputs longer than 90 minutes with up to four distinct speakers.

Q: How does VibeVoice clone a voice from a reference clip?

VibeVoice uses an uploaded audio sample as the speaker reference and generates new speech from a linked transcript. The demonstrations use reference clips lasting 22, 14, 9, 8, and four seconds, with the resulting speech retaining recognizable vocal qualities.

Q: Can VibeVoice generate conversations with multiple speakers?

VibeVoice supports up to four distinct speakers in one output. The demonstrations include two-speaker conversations and a three-speaker meeting in which separate reference voices are assigned to designated transcript lines.

Q: How does VibeVoice add emotion to generated speech?

VibeVoice applies expression according to the context of the written transcript. In one test, the generated delivery shifts from happiness and excitement to disappointment and sadness before ending in anger, without separate emotion instructions.

Q: What languages does VibeVoice support in the demonstrations?

The demonstrations include Japanese, Spanish, German, Chinese, Korean, and English speech. A mixed-language test combines English, Chinese, and Korean in one passage, although the presenter says he could not personally verify the German pronunciation.

Q: Can VibeVoice preserve accents and distinctive vocal traits?

The presented tests retain qualities such as accent, tone, pitch, and expressive speaking style from the reference recordings. Examples include a British male voice, a chipmunk-like voice, and English spoken with a Japanese accent from a Japanese reference voice.

Q: Can VibeVoice create long podcasts and audiobooks?

VibeVoice can generate outputs longer than 90 minutes and include up to four distinct speakers. The tutorial identifies podcast episodes and audiobooks as suitable uses for these long, multi-speaker generations.

Summary & Key Takeaways

• VibeVoice is presented as a free, open-source voice cloning and text-to-speech system from Microsoft. It can be downloaded for offline, unlimited use and can produce recordings longer than 90 minutes. Support for as many as four distinct speakers makes it suitable for creating extended podcast episodes, conversations, and audiobooks.

• The demonstrations test voice cloning with reference recordings as short as four seconds. Generated conversations preserve recognizable vocal qualities, accents, tones, and unusual character traits. Examples include two-person arguments, a three-speaker meeting, public-figure voices, a British voice, a chipmunk-like voice, and an especially expressive fictional character voice.

• VibeVoice applies expressions according to the written context, shifting among excitement, sadness, disappointment, and anger without separate emotion instructions. It also produces Japanese, Spanish, German, Chinese, Korean, and English speech in the demonstrations. The tutorial covers online access, local installation through ComfyUI, and operation inside the ComfyUI workflow.