![\"Bar](\"https://huggingface.co/datasets/olesiagradium/video/resolve/main/phonon-ttfa-chart.png\")
Two weeks ago, we opened up Phonon, our small text-to-speech model that runs on your own device. It needs no server and no network, and what you type stays with you.
\nWe've been very happy with the reception so far. So here's the story of the launch: why we built Phonon, what it can do, the demos that came with it, and how it did on Hugging Face's Open TTS Leaderboard.
\nMost voice AI lives in the cloud: every sentence makes a round trip to a server. Phonon runs on your own device instead. Your text stays there and the voice works without a network. We’ve also opened the weights. Phonon 31m is Apache 2.0, so you can use it for free, even in a commercial product. The other models are CC BY-NC, so they’re free for non-commercial projects. If you’re building something commercial with those, pricing is based on your number of users, not how much speech you generate.
\nGetting a good voice to run on a small device has always been hard. You could make the model small, but then it didn’t sound great. Or you could make it sound great, but then it wouldn’t fit. We addressed both.
\nWe announced Phonon in April and spent the next six months building on it with partners and early users. Their feedback helped us to shape this release. The models got smaller, and we added more languages and support for custom voices through voice cloning and voice design.
\nPhonon is based on the Pocket TTS lineage and as such uses flow matching to generate the audio which is very different from the modeling techniques used in other TTS systems. This is what allowed us to bring this level of quality to such a small model but it created new challenges while training the models. The training was initially numerically unstable and required several tricks to produce quality results on all five languages.
\nPhonon comes in two sizes, at about 65 MB and 75 MB. Together they speak English, French, German, Spanish and Portuguese. There are more than 100 built-in voices, and you can create your own in Gradium Studio. Phonon also handles the important things that most other TTS systems often stumble on, like phone numbers, dates and codes.
\nIt runs on phones, laptops, single-board computers and in a browser tab, so you can hear it right now. Phonon 31m speaks English and is Apache 2.0, so you can build on it freely, including for commercial products. Phonon 40m covers all five languages under a non-commercial license.
\nThe launch came with demos from friends and from our own team. On an Apple Watch with Wi-Fi off, Phonon talks you through a coffee brew step by step. On an iPhone with no signal, it counts your reps and keeps you going. A robot called Lamp, from Autonomous, wears a sorting hat and talks in character. In a browser, an offline dino runner becomes a radio show hosted by Rex. There's also a sci-fi game voiced on the player's own machine, and Proust read aloud page by page.
\nThe Hugging Face team built the Transformers integration, so developers could use Phonon in Transformers from day one.
\nDevelopers have started building on it too. An ExecuTorch conversion runs Phonon on a visitor's GPU in the browser, Micdrop brings it to Node.js apps, and Spaces like smallest-s2s build on it.
\nPhonon was downloaded more than 200,000 times in its first two weeks, and it’s still trending on Hugging Face.
\nThe Open TTS Leaderboard is a very new project from Hugging Face, and we love it. With more than 9,000 TTS models on the Hugging Face Hub, the leaderboard gives developers one place to compare open models and hear them side by side. Across 23 languages, models are evaluated on intelligibility (word error rate of the outputs), speed (inverse real-time factor and time-to-first audio), and naturalness (UTMOS for English and KOMOS for Korean). Fixed-voice and voice cloning are evaluated, with an additional speaker similarity metric for the latter. The leaderboard also has a Listen tab, where you can conveniently compare TTS outputs yourself and vote for the ones you like the most.
\nOn intelligibility, Phonon ranks first in German and Spanish, and in the top three in every language we evaluated, next to models up to 100 times its size.
\nSpeed is where we see Phonon truly shine, as the models top the CPU streaming benchmark on time-to-first audio.
\n
We built a new inference framework, ptts, specifically for Phonon. The goal of ptts is to get the best performance on a wide variety of devices and as such supports more than 10 backends: CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU), and NPU (Apple Core ML, Snapdragon®).
\nPhonon is also available in 🤗 Transformers, from v5.21, as its reference PyTorch implementation. Where ptts runs a single stream on device, Transformers targets GPU throughput: batched generation, full-graph torch.compile, e.g., for synthesizing speech datasets at scale. Being native PyTorch, it is a base to build upon, whether to integrate Phonon into your own stack or to study the model itself.
It belongs to the Pocket TTS lineage, so we first added Pocket TTS to Transformers. Phonon sits on top as a modular model: about 200 lines describing only what differs.
\nPhonon generates audio latents autoregressively: each step outputs a latent that is fed back as the next input, and the training loss is computed on them. Latents are the model's inputs, outputs and targets, as text tokens are for a language model, so the codec mapping audio to latents plays the role of a tokenizer. Transformers ships it as such: an audio tokenizer, held by the processor next to the text tokenizer:
\n\n\n = \n = \n = \n\n # 30,788,354\n # 10,305,313\nA voice is not a recording but a tensor of speaker latents. Phonon never encodes audio directly with its audio tokenizer, which ships decoder-only, and voices come precomputed: gradium/phonon-voices.
\n\n\n\n = \n = . # (num_frames, 512)\nThe prompt layout (voice positions, text, then a token opening the audio) ships with the checkpoint as a chat template, as for other multimodal models in Transformers. You write a standard conversation, with an empty audio item where the voice goes, and your code stays the same across checkpoints. The processor then expands the voice placeholder to one position per frame, normalizes and tokenizes the text, and batches the inputs:
= \n = \n# prompt: <|audio_prompt|> × num_frames, "Hello from Phonon.", <|audio_bos|>\ngenerate returns latents, which the processor decodes to audio:
= # audio_latents: (batch_size, num_frames, 32)\n = \n\nSee the docs 🤗 for more!
\nPhonon can speak with any voice. Clone one from a short recording with the speaker’s consent, or describe a new one in plain text in Gradium Studio. Then download the voice file, load it in Phonon, and it runs fully on the device.
\nWe also train custom Phonon models with design partners. If your memory budget is tight, we can make the model smaller. If you need Korean, Arabic or Japanese, we can train for that too. And if your product has its own vocabulary, we can fine-tune on it. Get in touch and tell us what you need.
\nThank you to the Hugging Face team, and to everyone who tried Phonon in its first two weeks.
\nPhonon is our first open release. Try it out and tell us what you build.
\n", "content_markdown": "Two weeks ago, we opened up Phonon, our small text-to-speech model that runs on your own device. It needs no server and no network, and what you type stays with you.\n\nWe've been very happy with the reception so far. So here's the story of the launch: why we built Phonon, what it can do, the demos that came with it, and how it did on Hugging Face's Open TTS Leaderboard.\n\n## Why a voice should live on your device\n\nMost voice AI lives in the cloud: every sentence makes a round trip to a server. Phonon runs on your own device instead. Your text stays there and the voice works without a network. We’ve also opened the weights. Phonon 31m is Apache 2.0, so you can use it for free, even in a commercial product. The other models are CC BY-NC, so they’re free for non-commercial projects. If you’re building something commercial with those, pricing is based on your number of users, not how much speech you generate.\n\nGetting a good voice to run on a small device has always been hard. You could make the model small, but then it didn’t sound great. Or you could make it sound great, but then it wouldn’t fit. We addressed both.\n\nWe announced Phonon in April and spent the next six months building on it with partners and early users. Their feedback helped us to shape this release. The models got smaller, and we added more languages and support for custom voices through voice cloning and voice design.\n\nPhonon is based on the [Pocket TTS](https://huggingface.co/kyutai/pocket-tts) lineage and as such uses flow matching to generate the audio which is very different from the modeling techniques used in other TTS systems. This is what allowed us to bring this level of quality to such a small model but it created new challenges while training the models. The training was initially numerically unstable and required several tricks to produce quality results on all five languages.\n\n## What we released\n\nPhonon comes in two sizes, at about 65 MB and 75 MB. Together they speak English, French, German, Spanish and Portuguese. There are more than 100 built-in voices, and you can create your own in [Gradium Studio](https://studio.gradium.ai/). Phonon also handles the important things that most other TTS systems often stumble on, like phone numbers, dates and codes.\n\nIt runs on phones, laptops, single-board computers and in a browser tab, so you can [hear it right now](https://huggingface.co/spaces/gradium/phonon). Phonon 31m speaks English and is Apache 2.0, so you can build on it freely, including for commercial products. Phonon 40m covers all five languages under a non-commercial license.\n\n## Demos and first projects\n\n[The launch](https://gradium.ai/blog/release-phonon-open-weight) came with demos from friends and from our own team. On an Apple Watch with Wi-Fi off, Phonon talks you through a coffee brew step by step. On an iPhone with no signal, it counts your reps and keeps you going. A robot called Lamp, from Autonomous, wears a sorting hat and talks in character. In a browser, an offline dino runner becomes a radio show hosted by Rex. There's also a sci-fi game voiced on the player's own machine, and Proust read aloud page by page.\n\nThe Hugging Face team built the Transformers integration, so developers could use Phonon in Transformers from day one.\n\nDevelopers have started building on it too. An [ExecuTorch conversion](https://huggingface.co/executorch-community/Phonon-31M-EN-ExecuTorch-WebGPU) runs Phonon on a visitor's GPU in the browser, [Micdrop](https://micdrop.dev/docs/ai-integration/provided-integrations/phonon.md) brings it to Node.js apps, and Spaces like [smallest-s2s](https://huggingface.co/spaces/Steveeeeeeen/smallest-s2s) build on it.\n\nPhonon was downloaded more than 200,000 times in its first two weeks, and it’s still trending on Hugging Face.\n\n## How Phonon did on the Hugging Face Open TTS Leaderboard\n\nThe [Open TTS Leaderboard](https://huggingface.co/spaces/hf-audio/open_tts_leaderboard) is a very new project from Hugging Face, and we love it. With more than 9,000 TTS models on the [Hugging Face Hub](https://huggingface.co/models?pipeline_tag=text-to-speech), the leaderboard gives developers one place to compare open models and hear them side by side. Across 23 languages, models are evaluated on **intelligibility** (*word error rate of the outputs*), **speed** (*inverse real-time factor and time-to-first audio*), and **naturalness** (*UTMOS for English and KOMOS for Korean*). Fixed-voice and voice cloning are evaluated, with an additional speaker similarity metric for the latter. The leaderboard also has a Listen tab, where you can conveniently compare TTS outputs yourself and vote for the ones you like the most.\n\nOn intelligibility, Phonon ranks first in [German](https://huggingface.co/spaces/hf-audio/open_tts_leaderboard?languages=de) and [Spanish](https://huggingface.co/spaces/hf-audio/open_tts_leaderboard?languages=es), and in the top three in every language we evaluated, next to models up to 100 times its size.\n\nSpeed is where we see Phonon truly shine, as the models top the [CPU streaming benchmark](https://huggingface.co/spaces/hf-audio/open_tts_leaderboard?streaming-cpu) on time-to-first audio.\n\n[](https://huggingface.co/datasets/olesiagradium/video/resolve/main/phonon-ttfa-chart.png)\n\n## Inference\n\nWe built a new inference framework, [ptts](https://github.com/gradium-ai/xn-ptts), specifically for Phonon. The goal of ptts is to get the best performance on a wide variety of devices and as such supports more than 10 backends: CPU (Intel, Arm, WebAssembly), GPU (CUDA, Vulkan, Metal, WebGPU), and NPU (Apple Core ML, Snapdragon®).\n\nPhonon is also available in 🤗 Transformers, from v5.21, as its reference PyTorch implementation. Where `ptts` runs a single stream on device, Transformers targets GPU throughput: batched generation, full-graph `torch.compile`, e.g., for synthesizing speech datasets at scale. Being native PyTorch, it is a base to build upon, whether to integrate Phonon into your own stack or to study the model itself.\n\nIt belongs to the Pocket TTS lineage, so we first added Pocket TTS to Transformers. Phonon sits on top as a [modular](https://huggingface.co/docs/transformers/modular_transformers) model: about [200 lines](https://github.com/huggingface/transformers/blob/main/src/transformers/models/gradium_phonon_tts/modular_gradium_phonon_tts.py) describing only what differs.\n\nPhonon generates audio latents autoregressively: each step outputs a latent that is fed back as the next input, and the training loss is computed on them. Latents are the model's inputs, outputs and targets, as text tokens are for a language model, so the codec mapping audio to latents plays the role of a tokenizer. Transformers ships it as such: an [audio tokenizer](https://huggingface.co/docs/transformers/model_doc/pocket_tts_audio_tokenizer), held by the processor next to the text tokenizer:\n\n```py\nfrom transformers import AutoProcessor, GradiumPhononTTSForConditionalGeneration\n\nmodel_id = \"gradium/phonon-31m-en\"\nprocessor = AutoProcessor.from_pretrained(model_id, device_map=\"auto\")\nmodel = GradiumPhononTTSForConditionalGeneration.from_pretrained(model_id, device_map=\"auto\")\n\nprint(f\"{model.num_parameters():,}\") # 30,788,354\nprint(f\"{processor.audio_tokenizer.num_parameters():,}\") # 10,305,313\n```\n\nA voice is not a recording but a tensor of speaker latents. Phonon never encodes audio directly with its audio tokenizer, which ships decoder-only, and voices come precomputed: [gradium/phonon-voices](https://huggingface.co/gradium/phonon-voices).\n\n```py\nfrom huggingface_hub import hf_hub_download\nfrom safetensors.torch import load_file\n\npath = hf_hub_download(\"gradium/phonon-voices\", \"voices/LMDIYE0Oa377Qh6Z.safetensors\")\nvoice = load_file(path)[\"speaker_wavs\"][0].T # (num_frames, 512)\n```\n\nThe prompt layout (voice positions, text, then a token opening the audio) ships with the checkpoint as a chat template, as for other multimodal models in Transformers. You write a standard conversation, with an empty `audio` item where the voice goes, and your code stays the same across checkpoints. The processor then expands the voice placeholder to one position per frame, normalizes and tokenizes the text, and batches the inputs:\n\n```py\nconversation = [\n {\"role\": \"user\", \"content\": [{\"type\": \"audio\"}, {\"type\": \"text\", \"text\": \"Hello from Phonon.\"}]}\n]\ninputs = processor.apply_chat_template(\n conversation,\n tokenize=True,\n return_dict=True,\n add_generation_prompt=True,\n processor_kwargs={\"audio_prompt_latents\": [voice]},\n).to(model.device)\n# prompt: <|audio_prompt|> × num_frames, \"Hello from Phonon.\", <|audio_bos|>\n```\n\n`generate` returns latents, which the processor decodes to audio:\n\n```py\noutputs = model.generate(**inputs) # audio_latents: (batch_size, num_frames, 32)\naudio = processor.decode(outputs.audio_latents, outputs.audio_latents_mask)\nprocessor.save_audio(audio, \"phonon.wav\")\n```\n\nSee the [docs](https://huggingface.co/docs/transformers/model_doc/gradium_phonon_tts) 🤗 for more!\n\n## What's next\n\nPhonon can speak with any voice. Clone one from a short recording with the speaker’s consent, or describe a new one in plain text in [Gradium Studio](https://studio.gradium.ai/). Then download the voice file, load it in Phonon, and it runs fully on the device.\n\nWe also train custom Phonon models with design partners. If your memory budget is tight, we can make the model smaller. If you need Korean, Arabic or Japanese, we can train for that too. And if your product has its own vocabulary, we can fine-tune on it. [Get in touch](https://gradium.ai/contact?subject=Sales) and tell us what you need.\n\nThank you to the Hugging Face team, and to everyone who tried Phonon in its first two weeks.\n\nPhonon is our first open release. [Try it out](https://huggingface.co/spaces/gradium/phonon) and tell us what you build.\n", "dateCreated": "2026-11-05T17:57:26.867095589+00:00", "date_created": "2026-11-05T00:00:00+00:00", "date_modified": "2026-11-05T00:00:00+00:00", "date_published": "2026-11-05T00:00:00+00:00", "description": "Two weeks ago, we opened up Phonon, our small text-to-speech model that runs on your own device. It needs no server and no network, and what you type stays with you. We've been very happy with the reception so far. So here's the story of…", "external_url": "https://huggingface.co/blog/phonon-tts", "git": { "commit": "20068632b65d2830a2d54f25ca1d90d697bea2c0", "path": "items/huggingface-co/2026/11/2026-11-05-a-closer-look-at-phonon-tts-launching-high-quality-on.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/20068632b65d2830a2d54f25ca1d90d697bea2c0/items/huggingface-co/2026/11/2026-11-05-a-closer-look-at-phonon-tts-launching-high-quality-on.md" }, "id": "urn:aggr:item:44f87ce4256c4d3f191a1e765fb56f53364301f8", "isBasedOn": { "@id": "https://huggingface.co/blog/phonon-tts" }, "mainEntity": { "@id": "https://huggingface.co/blog/phonon-tts", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-11-05-a-closer-look-at-phonon-tts-launching-high-quality-on/#webpage" }, "dateModified": "2026-11-05T00:00:00+00:00", "datePublished": "2026-11-05T00:00:00+00:00", "headline": "A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.", "image": { "@type": "ImageObject", "caption": null, "height": 90, "url": "https://huggingface.co/blog/assets/phonon-closer-look/thumbnail.png", "width": 160 }, "timeRequired": "PT6M", "url": "https://huggingface.co/blog/phonon-tts", "wordCount": 1347 }, "name": "Archived snapshot: A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.", "preview": { "alt": null, "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://huggingface.co/blog/assets/phonon-closer-look/thumbnail.png", "width": 160 }, "reading_minutes": 6, "replicated_at": null, "source": "huggingface-co", "source_name": "Hugging Face - Blog", "summary": null, "tags": [], "temporalCoverage": "2026-11-05T17:57:26.867095589+00:00", "title": "A Closer Look at Phonon TTS. Launching High-Quality On-Device Speech That Tops the Benchmarks.", "url": "https://aggr.aymericbeaumet.com/items/huggingface-co/2026-11-05-a-closer-look-at-phonon-tts-launching-high-quality-on/", "word_count": 1347 }