Open Source Models for Voice AI

DeepSeek-V3 — DeepSeek-V3 — open source.

Whisper — Robust Speech Recognition via Large-Scale Weak Supervision

Real-Time Voice Cloning — Clone a voice in 5 seconds to generate arbitrary speech in real-time

Llama — Inference code for Llama models

VibeVoice — Open-Source Frontier Voice AI

Coqui TTS — 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

ChatTTS — A generative speech model for daily dialogue.

Myshell — Instant voice cloning by MIT and MyShell. Audio foundation model.

Voicebox — Open Source Voice Cloning Desktop App for Mac, Windows, and Linux.

fish-speech — SOTA Open Source TTS

Fish Audio — Open-source AI voice platform with high-quality TTS and voice cloning in multiple languages.

DeepSpeech — DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Resemble AI — SoTA open-source TTS

CosyVoice — Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

LiveKit — Open source platform for building, testing, deploying, and scaling voice, video, and physical AI agents.

NVIDIA NeMo — A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

Kaldi — kaldi-asr/kaldi is the official location of the Kaldi project.

Vosk — Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node

F5-TTS — Official code for "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching"

Pipecat — Open source framework for building multi-modal conversational AI with voice capabilities.

SpeechBrain — A PyTorch-based Speech Toolkit

pyannote.audio — Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

ESPnet — End-to-End Speech Processing Toolkit

Silero VAD — Silero VAD: pre-trained enterprise-grade Voice Activity Detector

Moonshine — Very low latency speech to text, intent recognition, and text to speech, for building voice agents and interfaces

kokoro — https://hf.co/hexgrad/Kokoro-82M

Myshell — High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.

Zonos — Zonos-v0.1 is a leading open-weight text-to-speech model trained on more than 200k hours of varied multilingual speech, delivering expressiveness and quality on par with—or even surpassing—top TTS providers.

neutts — On-device TTS model by Neuphonic

Gemma — Gemma open-weight LLM library, from Google DeepMind