{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/kimi-com/2025-04-26-kimi-audio/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_logo.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_logo.png)\n\nKimi-Audio-7B [🤗](https://huggingface.co/moonshotai/Kimi-Audio-7B)  | Kimi-Audio-7B-Instruct [🤗](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct)  | 📑 [Paper](https://arxiv.org/pdf/2504.18425)\n\nWe present Kimi-Audio, an open-source audio foundation model excelling in **audio understanding, generation, and conversation**. This repository contains the official implementation, models, and evaluation toolkit for Kimi-Audio.\n\n## 🔥🔥🔥 News!!\n\n- May 29, 2025: 👋 We release a finetuning example of [Kimi-Audio-7B](https://github.com/MoonshotAI/Kimi-Audio/tree/master/finetune_codes).\n- April 27, 2025: 👋 We release pretrained model weights of [Kimi-Audio-7B](https://huggingface.co/moonshotai/Kimi-Audio-7B).\n- April 25, 2025: 👋 We release the inference code and model weights of [Kimi-Audio-7B-Instruct](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct).\n- April 25, 2025: 👋 We release the audio evaluation toolkit [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit). We can easily reproduce the **our results and baselines** by this toolkit!\n- April 25, 2025: 👋 We release the technical report of [Kimi-Audio](https://arxiv.org/pdf/2504.18425).\n\n## Table of Contents\n\n- [Introduction](https://github.com/MoonshotAI/Kimi-Audio#introduction)\n- [Architecture Overview](https://github.com/MoonshotAI/Kimi-Audio#architecture-overview)\n- [Quick Start](https://github.com/MoonshotAI/Kimi-Audio#quick-start)\n- [Evaluation](https://github.com/MoonshotAI/Kimi-Audio#evaluation)\n - [Speech Recognition](https://github.com/MoonshotAI/Kimi-Audio#automatic-speech-recognition-asr)\n - [Audio Understanding](https://github.com/MoonshotAI/Kimi-Audio#audio-understanding)\n - [Audio-to-Text Chat](https://github.com/MoonshotAI/Kimi-Audio#audio-to-text-chat)\n - [Speech Conversation](https://github.com/MoonshotAI/Kimi-Audio#speech-conversation)\n- [Finetune](https://github.com/MoonshotAI/Kimi-Audio#finetune)\n- [Evaluation Toolkit](https://github.com/MoonshotAI/Kimi-Audio#evaluation-toolkit)\n- [Generation Testset](https://github.com/MoonshotAI/Kimi-Audio#generation-testset)\n- [License](https://github.com/MoonshotAI/Kimi-Audio#license)\n- [Acknowledgements](https://github.com/MoonshotAI/Kimi-Audio#acknowledgements)\n- [Citation](https://github.com/MoonshotAI/Kimi-Audio#citation)\n- [Contact Us](https://github.com/MoonshotAI/Kimi-Audio#contact-us)\n\n## Introduction\n\nKimi-Audio is designed as a universal audio foundation model capable of handling a wide variety of audio processing tasks within a single unified framework. Key features include:\n\n- **Universal Capabilities:** Handle diverse tasks like automatic speech recognition (ASR), audio question answering (AQA), automatic audio captioning (AAC), speech emotion recognition (SER), sound event/scene classification (SEC/ASC), and end-to-end speech conversation.\n- **State-of-the-Art Performance:** Achieve SOTA results on numerous audio benchmarks (see [Evaluation](https://github.com/MoonshotAI/Kimi-Audio#evaluation) and the [Technical Report](https://arxiv.org/pdf/2504.18425)).\n- **Large-Scale Pre-training:** Pre-train on over 13 million hours of diverse audio data (speech, music, sounds) and text data, enabling robust audio reasoning and language understanding.\n- **Novel Architecture:** Employ a hybrid audio input (continuous acoustic vectors + discrete semantic tokens) and an LLM core with parallel heads for text and audio token generation.\n- **Efficient Inference:** Feature a chunk-wise streaming detokenizer based on flow matching for low-latency audio generation.\n- **Open-Source:** Release the code and model checkpoints for both pre-training and instruction fine-tuning, and release a comprehensive evaluation toolkit to foster community research and development.\n\n## Architecture Overview\n\n[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_framework.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_framework.png)\n\nKimi-Audio consists of three main components:\n\n1. **Audio Tokenizer:** Converts input audio into:\n - Discrete semantic tokens (12.5Hz) using vector quantization.\n - Continuous acoustic features derived from a Whisper encoder (downsampled to 12.5Hz).\n2. **Audio LLM:** A transformer-based model (initialized from a pre-trained text LLM like Qwen 2.5 7B) with shared layers processing multimodal inputs, followed by parallel heads for autoregressively generating text tokens and discrete audio semantic tokens.\n3. **Audio Detokenizer:** Converts the predicted discrete semantic audio tokens back into high-fidelity waveforms using a flow-matching model and a vocoder (BigVGAN), supporting chunk-wise streaming with a look-ahead mechanism for low latency.\n\n## Getting Started\n\n### Step1: Get the Code\n\n```shell\ngit clone https://github.com/MoonshotAI/Kimi-Audio.git\ncd Kimi-Audio\ngit submodule update --init --recursive\npip install -r requirements.txt\n```\n\nKimi‑Audio can now be installed directly via **pip**.\n\n```shell\npip install torch\npip install git+https://github.com/MoonshotAI/Kimi-Audio.git\n```\n\n## Quick Start\n\nThis example demonstrates basic usage for generating text from audio (ASR) and generating both text and speech in a conversational turn.\n\n```python\nimport soundfile as sf\nfrom kimia_infer.api.kimia import KimiAudio\n\n# --- 1. Load Model ---\nmodel_path = \"moonshotai/Kimi-Audio-7B-Instruct\" \nmodel = KimiAudio(model_path=model_path, load_detokenizer=True)\n\n# --- 2. Define Sampling Parameters ---\nsampling_params = {\n \"audio_temperature\": 0.8,\n \"audio_top_k\": 10,\n \"text_temperature\": 0.0,\n \"text_top_k\": 5,\n \"audio_repetition_penalty\": 1.0,\n \"audio_repetition_window_size\": 64,\n \"text_repetition_penalty\": 1.0,\n \"text_repetition_window_size\": 16,\n}\n\n# --- 3. Example 1: Audio-to-Text (ASR) ---\nmessages_asr = [\n # You can provide context or instructions as text\n {\"role\": \"user\", \"message_type\": \"text\", \"content\": \"Please transcribe the following audio:\"},\n # Provide the audio file path\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/asr_example.wav\"}\n]\n\n# Generate only text output\n_, text_output = model.generate(messages_asr, **sampling_params, output_type=\"text\")\nprint(\">>> ASR Output Text: \", text_output) # Expected output: \"这并不是告别，这是一个篇章的结束，也是新篇章的开始。\"\n\n\n# --- 4. Example 2: Audio-to-Audio/Text Conversation ---\nmessages_conversation = [\n # Start conversation with an audio query\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/qa_example.wav\"}\n]\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type=\"both\")\n\n# Save the generated audio\noutput_audio_path = \"output_audio.wav\"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f\">>> Conversational Output Audio saved to: {output_audio_path}\")\nprint(\">>> Conversational Output Text: \", text_output) # Expected output: \"当然可以，这很简单。一二三四五六七八九十。\"\n\n# --- 5. Example 3: Audio-to-Audio/Text Conversation with Multiturn ---\n\nmessages = [\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/multiturn/case2/multiturn_q1.wav\"},\n # This is the first turn output of Kimi-Audio\n {\"role\": \"assistant\", \"message_type\": \"audio-text\", \"content\": [\"test_audios/multiturn/case2/multiturn_a1.wav\", \"当然可以，这很简单。一二三四五六七八九十。\"]},\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/multiturn/case2/multiturn_q2.wav\"}\n]\nwav, text = model.generate(messages, **sampling_params, output_type=\"both\")\n\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type=\"both\")\n\n# Save the generated audio\noutput_audio_path = \"output_audio.wav\"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f\">>> Conversational Output Audio saved to: {output_audio_path}\")\nprint(\">>> Conversational Output Text: \", text_output) # Expected output: \"没问题，继续数下去就是十一十二十三十四十五十六十七十八十九二十。\"\n\nprint(\"Kimi-Audio inference examples complete.\")\n```\n\n## Evaluation\n\nKimi-Audio achieves state-of-the-art (SOTA) performance across a wide range of audio benchmarks.\n\nThe below is the overall performance:\n\n[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_radar_chart.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_radar_chart.png)\n\nHere are performances on different benchmarks, you can easily reproduce the **our results and baselines** by our [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit) (also see [**Evaluation Toolkit**](https://github.com/MoonshotAI/Kimi-Audio#evaluation-toolkit)):\n\n### Automatic Speech Recognition (ASR)\n\n| Datasets | Model | Performance (WER↓) |\n| ----------------------------------------------- | ------------------- | ------------------ |\n| **LibriSpeech** test-clean | test-other | Qwen2-Audio-base | 1.74 | 4.04 |\n| Baichuan-base | 3.02 | 6.04 | |\n| Step-Audio-chat | 3.19 | 10.67 | |\n| Qwen2.5-Omni | 2.37 | 4.21 | |\n| Kimi-Audio | **1.28** | **2.42** | |\n| **Fleurs** zh | en | Qwen2-Audio-base | 3.63 | 5.20 |\n| Baichuan-base | 4.15 | 8.07 | |\n| Step-Audio-chat | 4.26 | 8.56 | |\n| Qwen2.5-Omni | 2.92 | **4.17** | |\n| Kimi-Audio | **2.69** | 4.44 | |\n| **AISHELL-1** | Qwen2-Audio-base | 1.52 |\n| Baichuan-base | 1.93 | |\n| Step-Audio-chat | 2.14 | |\n| Qwen2.5-Omni | 1.13 | |\n| Kimi-Audio | **0.60** | |\n| **AISHELL-2** ios | Qwen2-Audio-base | 3.08 |\n| Baichuan-base | 3.87 | |\n| Step-Audio-chat | 3.89 | |\n| Qwen2.5-Omni | **2.56** | |\n| Kimi-Audio | **2.56** | |\n| **WenetSpeech** test-meeting | test-net | Qwen2-Audio-base | 8.40 | 7.64 |\n| Baichuan-base | 13.28 | 10.13 | |\n| Step-Audio-chat | 10.83 | 9.47 | |\n| Qwen2.5-Omni | 7.71 | 6.04 | |\n| Kimi-Audio | **6.28** | **5.37** | |\n| **Kimi-ASR Internal Testset** subset1 | subset2 | Qwen2-Audio-base | 2.31 | 3.24 |\n| Baichuan-base | 3.41 | 5.60 | |\n| Step-Audio-chat | 2.82 | 4.74 | |\n| Qwen2.5-Omni | 1.53 | 2.68 | |\n| Kimi-Audio | **1.42** | **2.44** | |\n\n### Audio Understanding\n\n| Datasets | Model | Performance↑ |\n| ------------------------------- | ----------------------------- | --------------------- |\n| **MMAU** music | sound | speech | Qwen2-Audio-base | 58.98 | 69.07 | 52.55 |\n| Baichuan-chat | 49.10 | 59.46 | 42.47 | |\n| GLM-4-Voice | 38.92 | 43.54 | 32.43 | |\n| Step-Audio-chat | 49.40 | 53.75 | 47.75 | |\n| Qwen2.5-Omni | **62.16** | 67.57 | 53.92 | |\n| Kimi-Audio | 61.68 | **73.27** | **60.66** | |\n| **ClothoAQA** test | dev | Qwen2-Audio-base | 71.73 | 72.63 |\n| Baichuan-chat | 48.02 | 48.16 | |\n| Step-Audio-chat | 45.84 | 44.98 | |\n| Qwen2.5-Omni | **72.86** | 73.12 | |\n| Kimi-Audio | 71.24 | **73.18** | |\n| **VocalSound** | Qwen2-Audio-base | 93.82 |\n| Baichuan-base | 58.17 | |\n| Step-Audio-chat | 28.58 | |\n| Qwen2.5-Omni | 93.73 | |\n| Kimi-Audio | **94.85** | |\n| **Nonspeech7k** | Qwen2-Audio-base | 87.17 |\n| Baichuan-chat | 59.03 | |\n| Step-Audio-chat | 21.38 | |\n| Qwen2.5-Omni | 69.89 | |\n| Kimi-Audio | **93.93** | |\n| **MELD** | Qwen2-Audio-base | 51.23 |\n| Baichuan-chat | 23.59 | |\n| Step-Audio-chat | 33.54 | |\n| Qwen2.5-Omni | 49.83 | |\n| Kimi-Audio | **59.13** | |\n| **TUT2017** | Qwen2-Audio-base | 33.83 |\n| Baichuan-base | 27.9 | |\n| Step-Audio-chat | 7.41 | |\n| Qwen2.5-Omni | 43.27 | |\n| Kimi-Audio | **65.25** | |\n| **CochlScene** test | dev | Qwen2-Audio-base | 52.69 | 50.96 |\n| Baichuan-base | 34.93 | 34.56 | |\n| Step-Audio-chat | 10.06 | 10.42 | |\n| Qwen2.5-Omni | 63.82 | 63.82 | |\n| Kimi-Audio | **79.84** | **80.99** | |\n\n### Audio-to-Text Chat\n\n| Datasets | Model | Performance↑ |\n| ----------------------------------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------- |\n| **OpenAudioBench** AlpacaEval | Llama Questions | Reasoning QA | TriviaQA | Web Questions | Qwen2-Audio-chat | 57.19 | 69.67 | 42.77 | 40.30 | 45.20 |\n| Baichuan-chat | 59.65 | 74.33 | 46.73 | 55.40 | 58.70 | |\n| GLM-4-Voice | 57.89 | 76.00 | 47.43 | 51.80 | 55.40 | |\n| StepAudio-chat | 56.53 | 72.33 | 60.00 | 56.80 | **73.00** | |\n| Qwen2.5-Omni | 72.76 | 75.33 | **63.76** | 57.06 | 62.80 | |\n| Kimi-Audio | **75.73** | **79.33** | 58.02 | **62.10** | 70.20 | |\n| **VoiceBench** AlpacaEval | CommonEval | SD-QA | MMSU | Qwen2-Audio-chat | 3.69 | 3.40 | 35.35 | 35.43 |\n| Baichuan-chat | 4.00 | 3.39 | 49.64 | 48.80 | |\n| GLM-4-Voice | 4.06 | 3.48 | 43.31 | 40.11 | |\n| StepAudio-chat | 3.99 | 2.99 | 46.84 | 28.72 | |\n| Qwen2.5-Omni | 4.33 | 3.84 | 57.41 | 56.38 | |\n| Kimi-Audio | **4.46** | **3.97** | **63.12** | **62.17** | |\n| **VoiceBench** OpenBookQA | IFEval | AdvBench | Avg | Qwen2-Audio-chat | 49.01 | 22.57 | 98.85 | 54.72 |\n| Baichuan-chat | 63.30 | 41.32 | 86.73 | 62.51 | |\n| GLM-4-Voice | 52.97 | 24.91 | 88.08 | 57.17 | |\n| StepAudio-chat | 31.87 | 29.19 | 65.77 | 48.86 | |\n| Qwen2.5-Omni | 79.12 | 53.88 | 99.62 | 72.83 | |\n| Kimi-Audio | **83.52** | **61.10** | **100.00** | **76.93** | |\n\n### Speech Conversation\n\nPerformance of Kimi-Audio and baseline models on speech conversation.\n\n| Model | Ability | | | | | |\n| --------------- | -------- | -------- | -------- | -------- | -------- | -------- |\n| GPT-4o | 4.21 | **3.65** | 4.05 | **3.87** | **4.54** | **4.06** |\n| Step-Audio-chat | 3.25 | 2.87 | 3.33 | 3.05 | 4.14 | 3.33 |\n| GLM-4-Voice | 3.83 | 3.51 | 3.77 | 3.07 | 4.04 | 3.65 |\n| GPT-4o-mini | 3.15 | 2.71 | 4.24 | 3.16 | 4.01 | 3.45 |\n| Kimi-Audio | **4.30** | 3.45 | **4.27** | 3.39 | 4.09 | 3.90 |\n\n## Finetune\n\nWe release the pre-trained model and the lightweight finetune codes. Please refer to the [finetune\\_codes/README.md](https://github.com/MoonshotAI/Kimi-Audio/blob/master/finetune_codes/README.md) for more details.\n\n## Evaluation Toolkit\n\nEvaluating and comparing audio foundation models is challenging due to inconsistent metrics, varying inference configurations, and a lack of standardized generation evaluation. To address this, we developed and open-sourced an **Evaluation Toolkit**.\n\nKey features:\n\n- Integrates Kimi-Audio and other recent audio LLMs.\n- Implements standardized metric calculation and integrates LLMs for intelligent judging (e.g., for AQA).\n- Provides a unified platform for side-by-side comparisons with shareable inference 'recipes' for reproducibility.\n- Includes a benchmark for evaluating speech conversation abilities (control, empathy, style).\n\nWe encourage the community to use and contribute to this toolkit to foster more reliable and comparable benchmarking. Find it here: [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit).\n\n## Generation Testset\n\nWe collect and release [Kimi-Audio-Generation-Testset](https://huggingface.co/datasets/moonshotai/Kimi-Audio-GenTest), which is designed to benchmark and evaluate the conversational capabilities of audio-based dialogue models. It consists of a collection of audio files containing various instructions and conversational prompts. The primary goal is to assess a model's ability to generate not just relevant, but also appropriately styled audio responses. The language in dataset is Chinese.\n\n## License\n\nThe model is based and modified from [Qwen 2.5-7B](https://github.com/QwenLM/Qwen2.5). Code derived from Qwen2.5-7B is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). Other parts of the code are licensed under the [MIT License](https://opensource.org/licenses/MIT).\n\n## Acknowledgements\n\nWe would like to thank the following projects and individuals for their contributions to the development of Kimi-Audio:\n\n- [Whisper](https://github.com/openai/whisper)\n- [Transformers](https://github.com/huggingface/transformers)\n- [BigVGAN](https://github.com/NVIDIA/BigVGAN)\n- [GLM-4-Voice](https://github.com/THUDM/GLM-4-Voice)\n- [Qwen](https://github.com/QwenLM/Qwen/tree/main)\n\nThank you to all the open-source projects for their contributions to this project!\n\n## Citation\n\nIf you find Kimi-Audio useful in your research or applications, please cite our technical report:\n\n```text-bibtex\n@misc{kimiteam2025kimiaudiotechnicalreport,\n title={Kimi-Audio Technical Report}, \n author={KimiTeam and Ding Ding and Zeqian Ju and Yichong Leng and Songxiang Liu and Tong Liu and Zeyu Shang and Kai Shen and Wei Song and Xu Tan and Heyi Tang and Zhengtao Wang and Chu Wei and Yifei Xin and Xinran Xu and Jianwei Yu and Yutao Zhang and Xinyu Zhou and Y. Charles and Jun Chen and Yanru Chen and Yulun Du and Weiran He and Zhenxing Hu and Guokun Lai and Qingcheng Li and Yangyang Liu and Weidong Sun and Jianzhou Wang and Yuzhi Wang and Yuefeng Wu and Yuxin Wu and Dongchao Yang and Hao Yang and Ying Yang and Zhilin Yang and Aoxiong Yin and Ruibin Yuan and Yutong Zhang and Zaida Zhou},\n year={2025},\n eprint={2504.18425},\n archivePrefix={arXiv},\n primaryClass={eess.AS},\n url={https://arxiv.org/abs/2504.18425}, \n}\n```\n\n## Contact Us\n\nFor questions, issues, or collaboration inquiries, please feel free to open an issue on GitHub.\n", "authors": [], "captured_at": "2026-10-19T01:26:53.998049939+00:00", "category": "ai", "content_html": "

\"\"

\n

Kimi-Audio-7B 🤗  | Kimi-Audio-7B-Instruct 🤗  | 📑 Paper

\n

We present Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation. This repository contains the official implementation, models, and evaluation toolkit for Kimi-Audio.

\n

🔥🔥🔥 News!!

\n\n

Table of Contents

\n\n

Introduction

\n

Kimi-Audio is designed as a universal audio foundation model capable of handling a wide variety of audio processing tasks within a single unified framework. Key features include:

\n\n

Architecture Overview

\n

\"\"

\n

Kimi-Audio consists of three main components:

\n
    \n
  1. Audio Tokenizer: Converts input audio into:\n\n
    2. \n
  2. Audio LLM: A transformer-based model (initialized from a pre-trained text LLM like Qwen 2.5 7B) with shared layers processing multimodal inputs, followed by parallel heads for autoregressively generating text tokens and discrete audio semantic tokens.
    3. \n
  3. Audio Detokenizer: Converts the predicted discrete semantic audio tokens back into high-fidelity waveforms using a flow-matching model and a vocoder (BigVGAN), supporting chunk-wise streaming with a look-ahead mechanism for low latency.
    4. \n
\n

Getting Started

\n

Step1: Get the Code

\n
git clone https://github.com/MoonshotAI/Kimi-Audio.git\ncd Kimi-Audio\ngit submodule update --init --recursive\npip install -r requirements.txt\n
\n

Kimi‑Audio can now be installed directly via pip.

\n
pip install torch\npip install git+https://github.com/MoonshotAI/Kimi-Audio.git\n
\n

Quick Start

\n

This example demonstrates basic usage for generating text from audio (ASR) and generating both text and speech in a conversational turn.

\n
import soundfile as sf\nfrom kimia_infer.api.kimia import KimiAudio\n\n# --- 1. Load Model ---\nmodel_path = "moonshotai/Kimi-Audio-7B-Instruct" \nmodel = KimiAudio(model_path=model_path, load_detokenizer=True)\n\n# --- 2. Define Sampling Parameters ---\nsampling_params = {\n    "audio_temperature": 0.8,\n    "audio_top_k": 10,\n    "text_temperature": 0.0,\n    "text_top_k": 5,\n    "audio_repetition_penalty": 1.0,\n    "audio_repetition_window_size": 64,\n    "text_repetition_penalty": 1.0,\n    "text_repetition_window_size": 16,\n}\n\n# --- 3. Example 1: Audio-to-Text (ASR) ---\nmessages_asr = [\n    # You can provide context or instructions as text\n    {"role": "user", "message_type": "text", "content": "Please transcribe the following audio:"},\n    # Provide the audio file path\n    {"role": "user", "message_type": "audio", "content": "test_audios/asr_example.wav"}\n]\n\n# Generate only text output\n_, text_output = model.generate(messages_asr, **sampling_params, output_type="text")\nprint(">>> ASR Output Text: ", text_output) # Expected output: "这并不是告别，这是一个篇章的结束，也是新篇章的开始。"\n\n\n# --- 4. Example 2: Audio-to-Audio/Text Conversation ---\nmessages_conversation = [\n    # Start conversation with an audio query\n    {"role": "user", "message_type": "audio", "content": "test_audios/qa_example.wav"}\n]\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type="both")\n\n# Save the generated audio\noutput_audio_path = "output_audio.wav"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f">>> Conversational Output Audio saved to: {output_audio_path}")\nprint(">>> Conversational Output Text: ", text_output) # Expected output: "当然可以，这很简单。一二三四五六七八九十。"\n\n# --- 5. Example 3: Audio-to-Audio/Text Conversation with Multiturn ---\n\nmessages = [\n    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q1.wav"},\n    # This is the first turn output of Kimi-Audio\n    {"role": "assistant", "message_type": "audio-text", "content": ["test_audios/multiturn/case2/multiturn_a1.wav", "当然可以，这很简单。一二三四五六七八九十。"]},\n    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q2.wav"}\n]\nwav, text = model.generate(messages, **sampling_params, output_type="both")\n\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type="both")\n\n# Save the generated audio\noutput_audio_path = "output_audio.wav"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f">>> Conversational Output Audio saved to: {output_audio_path}")\nprint(">>> Conversational Output Text: ", text_output) # Expected output: "没问题，继续数下去就是十一十二十三十四十五十六十七十八十九二十。"\n\nprint("Kimi-Audio inference examples complete.")\n
\n

Evaluation

\n

Kimi-Audio achieves state-of-the-art (SOTA) performance across a wide range of audio benchmarks.

\n

The below is the overall performance:

\n

\"\"

\n

Here are performances on different benchmarks, you can easily reproduce the our results and baselines by our Kimi-Audio-Evalkit (also see Evaluation Toolkit):

\n

Automatic Speech Recognition (ASR)

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
DatasetsModelPerformance (WER↓)
LibriSpeech test-clean | test-otherQwen2-Audio-base1.74 | 4.04
Baichuan-base3.02 | 6.04
Step-Audio-chat3.19 | 10.67
Qwen2.5-Omni2.37 | 4.21
Kimi-Audio1.28 | 2.42
Fleurs zh | enQwen2-Audio-base3.63 | 5.20
Baichuan-base4.15 | 8.07
Step-Audio-chat4.26 | 8.56
Qwen2.5-Omni2.92 | 4.17
Kimi-Audio2.69 | 4.44
AISHELL-1Qwen2-Audio-base1.52
Baichuan-base1.93
Step-Audio-chat2.14
Qwen2.5-Omni1.13
Kimi-Audio0.60
AISHELL-2 iosQwen2-Audio-base3.08
Baichuan-base3.87
Step-Audio-chat3.89
Qwen2.5-Omni2.56
Kimi-Audio2.56
WenetSpeech test-meeting | test-netQwen2-Audio-base8.40 | 7.64
Baichuan-base13.28 | 10.13
Step-Audio-chat10.83 | 9.47
Qwen2.5-Omni7.71 | 6.04
Kimi-Audio6.28 | 5.37
Kimi-ASR Internal Testset subset1 | subset2Qwen2-Audio-base2.31 | 3.24
Baichuan-base3.41 | 5.60
Step-Audio-chat2.82 | 4.74
Qwen2.5-Omni1.53 | 2.68
Kimi-Audio1.42 | 2.44
\n

Audio Understanding

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
DatasetsModelPerformance↑
MMAU music | sound | speechQwen2-Audio-base58.98 | 69.07 | 52.55
Baichuan-chat49.10 | 59.46 | 42.47
GLM-4-Voice38.92 | 43.54 | 32.43
Step-Audio-chat49.40 | 53.75 | 47.75
Qwen2.5-Omni62.16 | 67.57 | 53.92
Kimi-Audio61.68 | 73.27 | 60.66
ClothoAQA test | devQwen2-Audio-base71.73 | 72.63
Baichuan-chat48.02 | 48.16
Step-Audio-chat45.84 | 44.98
Qwen2.5-Omni72.86 | 73.12
Kimi-Audio71.24 | 73.18
VocalSoundQwen2-Audio-base93.82
Baichuan-base58.17
Step-Audio-chat28.58
Qwen2.5-Omni93.73
Kimi-Audio94.85
Nonspeech7kQwen2-Audio-base87.17
Baichuan-chat59.03
Step-Audio-chat21.38
Qwen2.5-Omni69.89
Kimi-Audio93.93
MELDQwen2-Audio-base51.23
Baichuan-chat23.59
Step-Audio-chat33.54
Qwen2.5-Omni49.83
Kimi-Audio59.13
TUT2017Qwen2-Audio-base33.83
Baichuan-base27.9
Step-Audio-chat7.41
Qwen2.5-Omni43.27
Kimi-Audio65.25
CochlScene test | devQwen2-Audio-base52.69 | 50.96
Baichuan-base34.93 | 34.56
Step-Audio-chat10.06 | 10.42
Qwen2.5-Omni63.82 | 63.82
Kimi-Audio79.84 | 80.99
\n

Audio-to-Text Chat

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
DatasetsModelPerformance↑
OpenAudioBench AlpacaEval | Llama Questions | Reasoning QA | TriviaQA | Web QuestionsQwen2-Audio-chat57.19 | 69.67 | 42.77 | 40.30 | 45.20
Baichuan-chat59.65 | 74.33 | 46.73 | 55.40 | 58.70
GLM-4-Voice57.89 | 76.00 | 47.43 | 51.80 | 55.40
StepAudio-chat56.53 | 72.33 | 60.00 | 56.80 | 73.00
Qwen2.5-Omni72.76 | 75.33 | 63.76 | 57.06 | 62.80
Kimi-Audio75.73 | 79.33 | 58.02 | 62.10 | 70.20
VoiceBench AlpacaEval | CommonEval | SD-QA | MMSUQwen2-Audio-chat3.69 | 3.40 | 35.35 | 35.43
Baichuan-chat4.00 | 3.39 | 49.64 | 48.80
GLM-4-Voice4.06 | 3.48 | 43.31 | 40.11
StepAudio-chat3.99 | 2.99 | 46.84 | 28.72
Qwen2.5-Omni4.33 | 3.84 | 57.41 | 56.38
Kimi-Audio4.46 | 3.97 | 63.12 | 62.17
VoiceBench OpenBookQA | IFEval | AdvBench | AvgQwen2-Audio-chat49.01 | 22.57 | 98.85 | 54.72
Baichuan-chat63.30 | 41.32 | 86.73 | 62.51
GLM-4-Voice52.97 | 24.91 | 88.08 | 57.17
StepAudio-chat31.87 | 29.19 | 65.77 | 48.86
Qwen2.5-Omni79.12 | 53.88 | 99.62 | 72.83
Kimi-Audio83.52 | 61.10 | 100.00 | 76.93
\n

Speech Conversation

\n

Performance of Kimi-Audio and baseline models on speech conversation.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ModelAbility
GPT-4o4.213.654.053.874.544.06
Step-Audio-chat3.252.873.333.054.143.33
GLM-4-Voice3.833.513.773.074.043.65
GPT-4o-mini3.152.714.243.164.013.45
Kimi-Audio4.303.454.273.394.093.90
\n

Finetune

\n

We release the pre-trained model and the lightweight finetune codes. Please refer to the finetune_codes/README.md for more details.

\n

Evaluation Toolkit

\n

Evaluating and comparing audio foundation models is challenging due to inconsistent metrics, varying inference configurations, and a lack of standardized generation evaluation. To address this, we developed and open-sourced an Evaluation Toolkit.

\n

Key features:

\n\n

We encourage the community to use and contribute to this toolkit to foster more reliable and comparable benchmarking. Find it here: Kimi-Audio-Evalkit.

\n

Generation Testset

\n

We collect and release Kimi-Audio-Generation-Testset, which is designed to benchmark and evaluate the conversational capabilities of audio-based dialogue models. It consists of a collection of audio files containing various instructions and conversational prompts. The primary goal is to assess a model's ability to generate not just relevant, but also appropriately styled audio responses. The language in dataset is Chinese.

\n

License

\n

The model is based and modified from Qwen 2.5-7B. Code derived from Qwen2.5-7B is licensed under the Apache 2.0 License. Other parts of the code are licensed under the MIT License.

\n

Acknowledgements

\n

We would like to thank the following projects and individuals for their contributions to the development of Kimi-Audio:

\n\n

Thank you to all the open-source projects for their contributions to this project!

\n

Citation

\n

If you find Kimi-Audio useful in your research or applications, please cite our technical report:

\n
@misc{kimiteam2025kimiaudiotechnicalreport,\n      title={Kimi-Audio Technical Report}, \n      author={KimiTeam and Ding Ding and Zeqian Ju and Yichong Leng and Songxiang Liu and Tong Liu and Zeyu Shang and Kai Shen and Wei Song and Xu Tan and Heyi Tang and Zhengtao Wang and Chu Wei and Yifei Xin and Xinran Xu and Jianwei Yu and Yutao Zhang and Xinyu Zhou and Y. Charles and Jun Chen and Yanru Chen and Yulun Du and Weiran He and Zhenxing Hu and Guokun Lai and Qingcheng Li and Yangyang Liu and Weidong Sun and Jianzhou Wang and Yuzhi Wang and Yuefeng Wu and Yuxin Wu and Dongchao Yang and Hao Yang and Ying Yang and Zhilin Yang and Aoxiong Yin and Ruibin Yuan and Yutong Zhang and Zaida Zhou},\n      year={2025},\n      eprint={2504.18425},\n      archivePrefix={arXiv},\n      primaryClass={eess.AS},\n      url={https://arxiv.org/abs/2504.18425}, \n}\n
\n

Contact Us

\n

For questions, issues, or collaboration inquiries, please feel free to open an issue on GitHub.

\n", "content_markdown": "[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_logo.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_logo.png)\n\nKimi-Audio-7B [🤗](https://huggingface.co/moonshotai/Kimi-Audio-7B)  | Kimi-Audio-7B-Instruct [🤗](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct)  | 📑 [Paper](https://arxiv.org/pdf/2504.18425)\n\nWe present Kimi-Audio, an open-source audio foundation model excelling in **audio understanding, generation, and conversation**. This repository contains the official implementation, models, and evaluation toolkit for Kimi-Audio.\n\n## 🔥🔥🔥 News!!\n\n- May 29, 2025: 👋 We release a finetuning example of [Kimi-Audio-7B](https://github.com/MoonshotAI/Kimi-Audio/tree/master/finetune_codes).\n- April 27, 2025: 👋 We release pretrained model weights of [Kimi-Audio-7B](https://huggingface.co/moonshotai/Kimi-Audio-7B).\n- April 25, 2025: 👋 We release the inference code and model weights of [Kimi-Audio-7B-Instruct](https://huggingface.co/moonshotai/Kimi-Audio-7B-Instruct).\n- April 25, 2025: 👋 We release the audio evaluation toolkit [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit). We can easily reproduce the **our results and baselines** by this toolkit!\n- April 25, 2025: 👋 We release the technical report of [Kimi-Audio](https://arxiv.org/pdf/2504.18425).\n\n## Table of Contents\n\n- [Introduction](https://github.com/MoonshotAI/Kimi-Audio#introduction)\n- [Architecture Overview](https://github.com/MoonshotAI/Kimi-Audio#architecture-overview)\n- [Quick Start](https://github.com/MoonshotAI/Kimi-Audio#quick-start)\n- [Evaluation](https://github.com/MoonshotAI/Kimi-Audio#evaluation)\n - [Speech Recognition](https://github.com/MoonshotAI/Kimi-Audio#automatic-speech-recognition-asr)\n - [Audio Understanding](https://github.com/MoonshotAI/Kimi-Audio#audio-understanding)\n - [Audio-to-Text Chat](https://github.com/MoonshotAI/Kimi-Audio#audio-to-text-chat)\n - [Speech Conversation](https://github.com/MoonshotAI/Kimi-Audio#speech-conversation)\n- [Finetune](https://github.com/MoonshotAI/Kimi-Audio#finetune)\n- [Evaluation Toolkit](https://github.com/MoonshotAI/Kimi-Audio#evaluation-toolkit)\n- [Generation Testset](https://github.com/MoonshotAI/Kimi-Audio#generation-testset)\n- [License](https://github.com/MoonshotAI/Kimi-Audio#license)\n- [Acknowledgements](https://github.com/MoonshotAI/Kimi-Audio#acknowledgements)\n- [Citation](https://github.com/MoonshotAI/Kimi-Audio#citation)\n- [Contact Us](https://github.com/MoonshotAI/Kimi-Audio#contact-us)\n\n## Introduction\n\nKimi-Audio is designed as a universal audio foundation model capable of handling a wide variety of audio processing tasks within a single unified framework. Key features include:\n\n- **Universal Capabilities:** Handle diverse tasks like automatic speech recognition (ASR), audio question answering (AQA), automatic audio captioning (AAC), speech emotion recognition (SER), sound event/scene classification (SEC/ASC), and end-to-end speech conversation.\n- **State-of-the-Art Performance:** Achieve SOTA results on numerous audio benchmarks (see [Evaluation](https://github.com/MoonshotAI/Kimi-Audio#evaluation) and the [Technical Report](https://arxiv.org/pdf/2504.18425)).\n- **Large-Scale Pre-training:** Pre-train on over 13 million hours of diverse audio data (speech, music, sounds) and text data, enabling robust audio reasoning and language understanding.\n- **Novel Architecture:** Employ a hybrid audio input (continuous acoustic vectors + discrete semantic tokens) and an LLM core with parallel heads for text and audio token generation.\n- **Efficient Inference:** Feature a chunk-wise streaming detokenizer based on flow matching for low-latency audio generation.\n- **Open-Source:** Release the code and model checkpoints for both pre-training and instruction fine-tuning, and release a comprehensive evaluation toolkit to foster community research and development.\n\n## Architecture Overview\n\n[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_framework.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_framework.png)\n\nKimi-Audio consists of three main components:\n\n1. **Audio Tokenizer:** Converts input audio into:\n - Discrete semantic tokens (12.5Hz) using vector quantization.\n - Continuous acoustic features derived from a Whisper encoder (downsampled to 12.5Hz).\n2. **Audio LLM:** A transformer-based model (initialized from a pre-trained text LLM like Qwen 2.5 7B) with shared layers processing multimodal inputs, followed by parallel heads for autoregressively generating text tokens and discrete audio semantic tokens.\n3. **Audio Detokenizer:** Converts the predicted discrete semantic audio tokens back into high-fidelity waveforms using a flow-matching model and a vocoder (BigVGAN), supporting chunk-wise streaming with a look-ahead mechanism for low latency.\n\n## Getting Started\n\n### Step1: Get the Code\n\n```shell\ngit clone https://github.com/MoonshotAI/Kimi-Audio.git\ncd Kimi-Audio\ngit submodule update --init --recursive\npip install -r requirements.txt\n```\n\nKimi‑Audio can now be installed directly via **pip**.\n\n```shell\npip install torch\npip install git+https://github.com/MoonshotAI/Kimi-Audio.git\n```\n\n## Quick Start\n\nThis example demonstrates basic usage for generating text from audio (ASR) and generating both text and speech in a conversational turn.\n\n```python\nimport soundfile as sf\nfrom kimia_infer.api.kimia import KimiAudio\n\n# --- 1. Load Model ---\nmodel_path = \"moonshotai/Kimi-Audio-7B-Instruct\" \nmodel = KimiAudio(model_path=model_path, load_detokenizer=True)\n\n# --- 2. Define Sampling Parameters ---\nsampling_params = {\n \"audio_temperature\": 0.8,\n \"audio_top_k\": 10,\n \"text_temperature\": 0.0,\n \"text_top_k\": 5,\n \"audio_repetition_penalty\": 1.0,\n \"audio_repetition_window_size\": 64,\n \"text_repetition_penalty\": 1.0,\n \"text_repetition_window_size\": 16,\n}\n\n# --- 3. Example 1: Audio-to-Text (ASR) ---\nmessages_asr = [\n # You can provide context or instructions as text\n {\"role\": \"user\", \"message_type\": \"text\", \"content\": \"Please transcribe the following audio:\"},\n # Provide the audio file path\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/asr_example.wav\"}\n]\n\n# Generate only text output\n_, text_output = model.generate(messages_asr, **sampling_params, output_type=\"text\")\nprint(\">>> ASR Output Text: \", text_output) # Expected output: \"这并不是告别，这是一个篇章的结束，也是新篇章的开始。\"\n\n\n# --- 4. Example 2: Audio-to-Audio/Text Conversation ---\nmessages_conversation = [\n # Start conversation with an audio query\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/qa_example.wav\"}\n]\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type=\"both\")\n\n# Save the generated audio\noutput_audio_path = \"output_audio.wav\"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f\">>> Conversational Output Audio saved to: {output_audio_path}\")\nprint(\">>> Conversational Output Text: \", text_output) # Expected output: \"当然可以，这很简单。一二三四五六七八九十。\"\n\n# --- 5. Example 3: Audio-to-Audio/Text Conversation with Multiturn ---\n\nmessages = [\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/multiturn/case2/multiturn_q1.wav\"},\n # This is the first turn output of Kimi-Audio\n {\"role\": \"assistant\", \"message_type\": \"audio-text\", \"content\": [\"test_audios/multiturn/case2/multiturn_a1.wav\", \"当然可以，这很简单。一二三四五六七八九十。\"]},\n {\"role\": \"user\", \"message_type\": \"audio\", \"content\": \"test_audios/multiturn/case2/multiturn_q2.wav\"}\n]\nwav, text = model.generate(messages, **sampling_params, output_type=\"both\")\n\n\n# Generate both audio and text output\nwav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type=\"both\")\n\n# Save the generated audio\noutput_audio_path = \"output_audio.wav\"\nsf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output\nprint(f\">>> Conversational Output Audio saved to: {output_audio_path}\")\nprint(\">>> Conversational Output Text: \", text_output) # Expected output: \"没问题，继续数下去就是十一十二十三十四十五十六十七十八十九二十。\"\n\nprint(\"Kimi-Audio inference examples complete.\")\n```\n\n## Evaluation\n\nKimi-Audio achieves state-of-the-art (SOTA) performance across a wide range of audio benchmarks.\n\nThe below is the overall performance:\n\n[![](https://github.com/MoonshotAI/Kimi-Audio/raw/master/assets/kimia_radar_chart.png)](https://github.com/MoonshotAI/Kimi-Audio/blob/master/assets/kimia_radar_chart.png)\n\nHere are performances on different benchmarks, you can easily reproduce the **our results and baselines** by our [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit) (also see [**Evaluation Toolkit**](https://github.com/MoonshotAI/Kimi-Audio#evaluation-toolkit)):\n\n### Automatic Speech Recognition (ASR)\n\n| Datasets | Model | Performance (WER↓) |\n| ----------------------------------------------- | ------------------- | ------------------ |\n| **LibriSpeech** test-clean | test-other | Qwen2-Audio-base | 1.74 | 4.04 |\n| Baichuan-base | 3.02 | 6.04 | |\n| Step-Audio-chat | 3.19 | 10.67 | |\n| Qwen2.5-Omni | 2.37 | 4.21 | |\n| Kimi-Audio | **1.28** | **2.42** | |\n| **Fleurs** zh | en | Qwen2-Audio-base | 3.63 | 5.20 |\n| Baichuan-base | 4.15 | 8.07 | |\n| Step-Audio-chat | 4.26 | 8.56 | |\n| Qwen2.5-Omni | 2.92 | **4.17** | |\n| Kimi-Audio | **2.69** | 4.44 | |\n| **AISHELL-1** | Qwen2-Audio-base | 1.52 |\n| Baichuan-base | 1.93 | |\n| Step-Audio-chat | 2.14 | |\n| Qwen2.5-Omni | 1.13 | |\n| Kimi-Audio | **0.60** | |\n| **AISHELL-2** ios | Qwen2-Audio-base | 3.08 |\n| Baichuan-base | 3.87 | |\n| Step-Audio-chat | 3.89 | |\n| Qwen2.5-Omni | **2.56** | |\n| Kimi-Audio | **2.56** | |\n| **WenetSpeech** test-meeting | test-net | Qwen2-Audio-base | 8.40 | 7.64 |\n| Baichuan-base | 13.28 | 10.13 | |\n| Step-Audio-chat | 10.83 | 9.47 | |\n| Qwen2.5-Omni | 7.71 | 6.04 | |\n| Kimi-Audio | **6.28** | **5.37** | |\n| **Kimi-ASR Internal Testset** subset1 | subset2 | Qwen2-Audio-base | 2.31 | 3.24 |\n| Baichuan-base | 3.41 | 5.60 | |\n| Step-Audio-chat | 2.82 | 4.74 | |\n| Qwen2.5-Omni | 1.53 | 2.68 | |\n| Kimi-Audio | **1.42** | **2.44** | |\n\n### Audio Understanding\n\n| Datasets | Model | Performance↑ |\n| ------------------------------- | ----------------------------- | --------------------- |\n| **MMAU** music | sound | speech | Qwen2-Audio-base | 58.98 | 69.07 | 52.55 |\n| Baichuan-chat | 49.10 | 59.46 | 42.47 | |\n| GLM-4-Voice | 38.92 | 43.54 | 32.43 | |\n| Step-Audio-chat | 49.40 | 53.75 | 47.75 | |\n| Qwen2.5-Omni | **62.16** | 67.57 | 53.92 | |\n| Kimi-Audio | 61.68 | **73.27** | **60.66** | |\n| **ClothoAQA** test | dev | Qwen2-Audio-base | 71.73 | 72.63 |\n| Baichuan-chat | 48.02 | 48.16 | |\n| Step-Audio-chat | 45.84 | 44.98 | |\n| Qwen2.5-Omni | **72.86** | 73.12 | |\n| Kimi-Audio | 71.24 | **73.18** | |\n| **VocalSound** | Qwen2-Audio-base | 93.82 |\n| Baichuan-base | 58.17 | |\n| Step-Audio-chat | 28.58 | |\n| Qwen2.5-Omni | 93.73 | |\n| Kimi-Audio | **94.85** | |\n| **Nonspeech7k** | Qwen2-Audio-base | 87.17 |\n| Baichuan-chat | 59.03 | |\n| Step-Audio-chat | 21.38 | |\n| Qwen2.5-Omni | 69.89 | |\n| Kimi-Audio | **93.93** | |\n| **MELD** | Qwen2-Audio-base | 51.23 |\n| Baichuan-chat | 23.59 | |\n| Step-Audio-chat | 33.54 | |\n| Qwen2.5-Omni | 49.83 | |\n| Kimi-Audio | **59.13** | |\n| **TUT2017** | Qwen2-Audio-base | 33.83 |\n| Baichuan-base | 27.9 | |\n| Step-Audio-chat | 7.41 | |\n| Qwen2.5-Omni | 43.27 | |\n| Kimi-Audio | **65.25** | |\n| **CochlScene** test | dev | Qwen2-Audio-base | 52.69 | 50.96 |\n| Baichuan-base | 34.93 | 34.56 | |\n| Step-Audio-chat | 10.06 | 10.42 | |\n| Qwen2.5-Omni | 63.82 | 63.82 | |\n| Kimi-Audio | **79.84** | **80.99** | |\n\n### Audio-to-Text Chat\n\n| Datasets | Model | Performance↑ |\n| ----------------------------------------------------------------------------------------- | ------------------------------------------------- | ------------------------------------- |\n| **OpenAudioBench** AlpacaEval | Llama Questions | Reasoning QA | TriviaQA | Web Questions | Qwen2-Audio-chat | 57.19 | 69.67 | 42.77 | 40.30 | 45.20 |\n| Baichuan-chat | 59.65 | 74.33 | 46.73 | 55.40 | 58.70 | |\n| GLM-4-Voice | 57.89 | 76.00 | 47.43 | 51.80 | 55.40 | |\n| StepAudio-chat | 56.53 | 72.33 | 60.00 | 56.80 | **73.00** | |\n| Qwen2.5-Omni | 72.76 | 75.33 | **63.76** | 57.06 | 62.80 | |\n| Kimi-Audio | **75.73** | **79.33** | 58.02 | **62.10** | 70.20 | |\n| **VoiceBench** AlpacaEval | CommonEval | SD-QA | MMSU | Qwen2-Audio-chat | 3.69 | 3.40 | 35.35 | 35.43 |\n| Baichuan-chat | 4.00 | 3.39 | 49.64 | 48.80 | |\n| GLM-4-Voice | 4.06 | 3.48 | 43.31 | 40.11 | |\n| StepAudio-chat | 3.99 | 2.99 | 46.84 | 28.72 | |\n| Qwen2.5-Omni | 4.33 | 3.84 | 57.41 | 56.38 | |\n| Kimi-Audio | **4.46** | **3.97** | **63.12** | **62.17** | |\n| **VoiceBench** OpenBookQA | IFEval | AdvBench | Avg | Qwen2-Audio-chat | 49.01 | 22.57 | 98.85 | 54.72 |\n| Baichuan-chat | 63.30 | 41.32 | 86.73 | 62.51 | |\n| GLM-4-Voice | 52.97 | 24.91 | 88.08 | 57.17 | |\n| StepAudio-chat | 31.87 | 29.19 | 65.77 | 48.86 | |\n| Qwen2.5-Omni | 79.12 | 53.88 | 99.62 | 72.83 | |\n| Kimi-Audio | **83.52** | **61.10** | **100.00** | **76.93** | |\n\n### Speech Conversation\n\nPerformance of Kimi-Audio and baseline models on speech conversation.\n\n| Model | Ability | | | | | |\n| --------------- | -------- | -------- | -------- | -------- | -------- | -------- |\n| GPT-4o | 4.21 | **3.65** | 4.05 | **3.87** | **4.54** | **4.06** |\n| Step-Audio-chat | 3.25 | 2.87 | 3.33 | 3.05 | 4.14 | 3.33 |\n| GLM-4-Voice | 3.83 | 3.51 | 3.77 | 3.07 | 4.04 | 3.65 |\n| GPT-4o-mini | 3.15 | 2.71 | 4.24 | 3.16 | 4.01 | 3.45 |\n| Kimi-Audio | **4.30** | 3.45 | **4.27** | 3.39 | 4.09 | 3.90 |\n\n## Finetune\n\nWe release the pre-trained model and the lightweight finetune codes. Please refer to the [finetune\\_codes/README.md](https://github.com/MoonshotAI/Kimi-Audio/blob/master/finetune_codes/README.md) for more details.\n\n## Evaluation Toolkit\n\nEvaluating and comparing audio foundation models is challenging due to inconsistent metrics, varying inference configurations, and a lack of standardized generation evaluation. To address this, we developed and open-sourced an **Evaluation Toolkit**.\n\nKey features:\n\n- Integrates Kimi-Audio and other recent audio LLMs.\n- Implements standardized metric calculation and integrates LLMs for intelligent judging (e.g., for AQA).\n- Provides a unified platform for side-by-side comparisons with shareable inference 'recipes' for reproducibility.\n- Includes a benchmark for evaluating speech conversation abilities (control, empathy, style).\n\nWe encourage the community to use and contribute to this toolkit to foster more reliable and comparable benchmarking. Find it here: [Kimi-Audio-Evalkit](https://github.com/MoonshotAI/Kimi-Audio-Evalkit).\n\n## Generation Testset\n\nWe collect and release [Kimi-Audio-Generation-Testset](https://huggingface.co/datasets/moonshotai/Kimi-Audio-GenTest), which is designed to benchmark and evaluate the conversational capabilities of audio-based dialogue models. It consists of a collection of audio files containing various instructions and conversational prompts. The primary goal is to assess a model's ability to generate not just relevant, but also appropriately styled audio responses. The language in dataset is Chinese.\n\n## License\n\nThe model is based and modified from [Qwen 2.5-7B](https://github.com/QwenLM/Qwen2.5). Code derived from Qwen2.5-7B is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). Other parts of the code are licensed under the [MIT License](https://opensource.org/licenses/MIT).\n\n## Acknowledgements\n\nWe would like to thank the following projects and individuals for their contributions to the development of Kimi-Audio:\n\n- [Whisper](https://github.com/openai/whisper)\n- [Transformers](https://github.com/huggingface/transformers)\n- [BigVGAN](https://github.com/NVIDIA/BigVGAN)\n- [GLM-4-Voice](https://github.com/THUDM/GLM-4-Voice)\n- [Qwen](https://github.com/QwenLM/Qwen/tree/main)\n\nThank you to all the open-source projects for their contributions to this project!\n\n## Citation\n\nIf you find Kimi-Audio useful in your research or applications, please cite our technical report:\n\n```text-bibtex\n@misc{kimiteam2025kimiaudiotechnicalreport,\n title={Kimi-Audio Technical Report}, \n author={KimiTeam and Ding Ding and Zeqian Ju and Yichong Leng and Songxiang Liu and Tong Liu and Zeyu Shang and Kai Shen and Wei Song and Xu Tan and Heyi Tang and Zhengtao Wang and Chu Wei and Yifei Xin and Xinran Xu and Jianwei Yu and Yutao Zhang and Xinyu Zhou and Y. Charles and Jun Chen and Yanru Chen and Yulun Du and Weiran He and Zhenxing Hu and Guokun Lai and Qingcheng Li and Yangyang Liu and Weidong Sun and Jianzhou Wang and Yuzhi Wang and Yuefeng Wu and Yuxin Wu and Dongchao Yang and Hao Yang and Ying Yang and Zhilin Yang and Aoxiong Yin and Ruibin Yuan and Yutong Zhang and Zaida Zhou},\n year={2025},\n eprint={2504.18425},\n archivePrefix={arXiv},\n primaryClass={eess.AS},\n url={https://arxiv.org/abs/2504.18425}, \n}\n```\n\n## Contact Us\n\nFor questions, issues, or collaboration inquiries, please feel free to open an issue on GitHub.\n", "dateCreated": "2026-10-19T01:26:53.998049939+00:00", "date_created": "2025-04-26T00:00:00+00:00", "date_modified": null, "date_published": "2025-04-26T00:00:00+00:00", "description": "2025-04-26", "external_url": "https://github.com/MoonshotAI/Kimi-Audio", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/kimi-com/2025/04/2025-04-26-kimi-audio.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/kimi-com/2025/04/2025-04-26-kimi-audio.md" }, "id": "urn:aggr:item:ec8df625def3c9440551795eb8a41a9109ac6c46", "isBasedOn": { "@id": "https://github.com/MoonshotAI/Kimi-Audio" }, "mainEntity": { "@id": "https://github.com/MoonshotAI/Kimi-Audio", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/kimi-com/2025-04-26-kimi-audio/#webpage" }, "datePublished": "2025-04-26T00:00:00+00:00", "headline": "Kimi-Audio", "image": { "@type": "ImageObject", "caption": "Kimi-Audio", "height": 90, "url": "https://kimi-file.kimi.ai/prod-chat-kimi/kfs/4/2/2026-03-25/1d71teed3v89kkegmsvo0?x-tos-process=image%2Fauto-orient%2C1%2Fstrip%2Fignore-error%2C1", "width": 160 }, "timeRequired": "PT9M", "url": "https://github.com/MoonshotAI/Kimi-Audio", "wordCount": 1871 }, "name": "Archived snapshot: Kimi-Audio", "preview": { "alt": "Kimi-Audio", "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://kimi-file.kimi.ai/prod-chat-kimi/kfs/4/2/2026-03-25/1d71teed3v89kkegmsvo0?x-tos-process=image%2Fauto-orient%2C1%2Fstrip%2Fignore-error%2C1", "width": 160 }, "reading_minutes": 9, "replicated_at": null, "source": "kimi-com", "source_name": "Kimi Research Blog | Moonshot AI", "summary": "2025-04-26", "tags": [], "temporalCoverage": "2026-10-19T01:26:53.998049939+00:00", "title": "Kimi-Audio", "url": "https://aggr.aymericbeaumet.com/items/kimi-com/2025-04-26-kimi-audio/", "word_count": 1871 }