Open Source Models for Voice AI

Hume AI — TADA is an open-source speech-language model from Hume AI.

Myshell — Instant voice cloning by MIT and MyShell. Audio foundation model.

DeepSeek-V3 — DeepSeek-V3 — open source.

LiveKit — Open source platform for building, testing, deploying, and scaling voice, video, and physical AI agents.

Rapida — Open-source Voice AI platform for contact centers, enterprises, and agencies with full deployment control.

VoiceMesh — A production-inspired reliability lab for real-time voice AI infrastructure — full VAD to STT to LLM to TTS pipeline with observability, event streaming, durable workflows, and billing.

VibeVoice — Open-Source Frontier Voice AI

MiniMax — MiniMax-M3 is an open-source large language model released by MiniMax, available on GitHub to run and build on.

FireRedTeam — FireRedASR is FireRedTeam's open-source automatic speech recognition (ASR) model for high-accuracy speech-to-text.

Dia2-2B — Dia2-2B by nari-labs — open-source model on Hugging Face.

Gradium — Build expressive, low latency voice AI with a unified API for text, speech, and cloning.

Llama — Inference code for Llama models

ESPnet — End-to-End Speech Processing Toolkit

Pype AI — Whispey is an open-source observability and analytics tool for voice AI agents.

Julius — Open-Source Large Vocabulary Continuous Speech Recognition Engine

Kyutai — Kyutai's open speech-to-text and text-to-speech models built on the Delayed Streams Modeling framework for streaming, low-latency speech.

Agora — Real-time voice AI platform enabling natural, low-latency conversations.

Patter — Open-source SDK for voice AI to connect any agent to real phone calls in 4 lines.

Fish Audio — Open-source AI voice platform with high-quality TTS and voice cloning in multiple languages.

Resemble AI — SoTA open-source TTS

pyannote.audio — Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

Moonshot AI — Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation

Silero VAD — Silero VAD: pre-trained enterprise-grade Voice Activity Detector

Myshell — High-quality multi-lingual text-to-speech library by MyShell.ai. Support English, Spanish, French, Chinese, Japanese and Korean.

Izwi — Voice AI runtime for private deployment with an OpenAI-compatible API.

Real-Time Voice Cloning — Clone a voice in 5 seconds to generate arbitrary speech in real-time

Kyutai — Moshi is a speech-text foundation model and full-duplex spoken-dialogue framework, using Mimi, a state-of-the-art streaming neural audio codec — enabling real-time, natural voice conversation.

Kyutai — Hibiki is a model for streaming (simultaneous) speech translation — it adapts in real time to produce a correct translation chunk by chunk, rather than waiting for the full utterance.

Pipecat — Open source framework for building multi-modal conversational AI with voice capabilities.

CosyVoice — Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.