{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-05-19-qwen3-5-livetranslate-from-sound-to-sight-from-word-to-right/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "![Qwen3.5-LiveTranslate Main Image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/main.png)\n\n[DashScope](https://www.alibabacloud.com/help/en/model-studio/qwen3-5-livetranslate-flash-realtime) [Demo](https://omni.qwen.ai/live-translate)\n\n**Qwen3.5-LiveTranslate-Flash** is the latest simultaneous interpretation model in the Qwen family, built on top of Qwen3.5-Omni. It delivers real-time, multimodal translation that not only hears and translates speech, but also sees and understands visual context to produce more accurate translations. Compared with its predecessor Qwen3-LiveTranslate, Qwen3.5-LiveTranslate-Flash brings major upgrades across language coverage, latency, voice cloning, and terminology handling, making it well-suited for international meetings, livestream localization, online classrooms, and business negotiations.\n\n### Key Highlights\n\n- **Massively expanded language coverage: understands 18 → 60 languages, speaks 10 → 29 languages.** The language support of input audio and output text has grown from 18 to 60, and output audio language support from 10 to 29, covering far more cross-lingual combinations to meet multilingual interpretation needs in international meetings, livestream localization, online classrooms, and business negotiations.\n\n- **Ultra-low latency: powered by Readable Unit technology, faster text and speech output.** A novel Readable Unit real-time translation technique achieves more aggressive streaming output while preserving translation readability and semantic consistency. Average speech-to-speech per-token latency is reduced to to 2.8 seconds, ideal for latency-sensitive scenarios such as livestreams, co-hosting, and press conferences.\n\n- **Real-time voice cloning: one sentence to start, instantly “interpret in your voice”.** During simultaneous interpretation, the system automatically replicates the speaker’s vocal characteristics, keeping the translated speech sounding like “the same person” across languages, enhancing immersion and identity consistency, especially critical for streamers, guests, and hosts.\n\n- **Hotword enhancement: proper nouns and industry terms “recognized right, written right, translated right”.** Built-in Hotword capability prioritizes the recognition and translation of names, places, brand names, product models, and industry terminology. Hotwords can be dynamically configured and updated in real time per scenario, significantly reducing terminology mistranslation risk, well-suited for technical launches, medical/legal/financial meetings, and enterprise training.\n\n## Performance\n\nWe evaluate Qwen3.5-LiveTranslate-Flash in both offline and real-time (streaming) settings.\n\n### Offline Translation\n\nOn public multilingual speech translation benchmarks (FLEURS, CoVoST2), Qwen3.5-LiveTranslate-Flash achieves higher translation accuracy than mainstream commerical large speech models, significantly surpasses its predecessor Qwen3-LiveTranslate-Flash, and delivers breakthroughs in both language coverage and translation quality.\n\nOverview English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_overview_en-xx.png)\n\nOverview X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_overview_xx-en.png)\n\nFLEURS English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_fleurs_en-xx.png)\n\nFLEURS X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_fleurs_xx-en.png)\n\nCoVoST2 English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_covost_en-xx.png)\n\nCoVoST2 X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_covost_xx-en.png)\n\n### Real-Time Translation\n\nWith the Readable Unit streaming strategy, Qwen3.5-LiveTranslate-Flash reduces first-token latency by **3.45 s** and per-token latency by **1.88 s** compared to Qwen3-LiveTranslate-Flash, achieving an average speech-to-speech per-token latency of **2.8 s**, with virtually no loss in translation quality.\n\nOverview Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/online_overview.png)\n\n## Model Architecture\n\nQwen3.5-LiveTranslate is a translation large model built on the Qwen3.5-Omni Thinker-Talker architecture. The Thinker receives interleaved visual and audio inputs and generates text translations, while the Talker takes the translated text and source audio to produce speech with crosslingual voice cloning. For real-time simultaneous interpretation, we adopt a chunk-wise streaming input mechanism and introduce Readable Unit tags to control speech synthesis granularity, effectively reducing interpretation latency. Meanwhile, dynamic crosslingual voice cloning enables the model to preserve the speaker’s original vocal characteristics during real-time translation.\n\n![Qwen3.5-LiveTranslate model architecture overview](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/model_arch_v3.png)\n\nQwen3.5-LiveTranslate model architecture overview\n\n## More Supported Languages\n\nCompared to Qwen3-LiveTranslate, Qwen3.5-LiveTranslate significantly expands language coverage. The support of input audio and output text grows from 18 to 60 languages, and output audio support from 10 to 29 languages, enabling a far wider range of cross-lingual translation combinations across global scenarios.\n\n| | Qwen3-LiveTranslate | Qwen3.5-LiveTranslate |\n| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Input Modality | Audio / Video | Audio / Video |\n| Inference Mode | Offline / Streaming | Offline / Streaming |\n| Voice Cloning | ✗ | ✓ (3 modes: pre-registered / clone-once / real-time) |\n| Hotwords | Up to 1,000 | Up to 1,000 |\n| Input Audio Languages & Output Text Languages | **18 languages** Chinese, English, Russian, French, German, Portuguese, Spanish, Italian, Indonesian, Korean, Japanese, Vietnamese, Thai, Arabic, Cantonese, Hindi, Greek, Turkish | **60 languages** Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Nynorsk, Odia, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, Vietnamese |\n| Output Audio Languages | **10 languages** Chinese, English, French, German, Russian, Italian, Spanish, Portuguese, Japanese, Korean | **29 languages** Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Filipino, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, Persian |\n\n## 🎬 See It in Action\n\n### International Meeting\n\nA multilingual business meeting where participants speak in different languages and switch between them mid-sentence. Qwen3.5-LiveTranslate handles code-switching, diverse accents, and domain-specific terminology in real time — delivering fluent, natural translations without missing a beat.\n\n### Traveling Abroad\n\nA real-world travel scenario powered by Qwen AI Glasses: a Chinese tourist orders food at a local restaurant in Thailand. The model performs live Thai-to-Chinese translation on-device, combining visual context from the menu with spoken dialogue to produce accurate, context-aware translations — making cross-language communication effortless on the go.\n\n### Livestream Scenarios\n\nE-commerce livestream translation scenario. Qwen3.5-LiveTranslate accurately translates product specifications and numerical information, ensuring precise cross-language delivery of product parameters.\n\n### Classical Chinese Translation\n\nA scene from *Romance of the Three Kingdoms* narrated in classical Chinese (文言文). Qwen3.5-LiveTranslate accurately interprets and translates archaic Chinese prose into modern English, demonstrating its ability to handle literary and historical language beyond everyday speech.\n\n### Visual Disambiguation\n\nQwen3.5-LiveTranslate leverages visual context to resolve translation ambiguities. When a word or phrase has multiple possible meanings, the model uses what it sees — on-screen text, objects, or scene context — to select the correct interpretation, producing translations that are both accurate and contextually grounded.\n\n## Using Qwen3.5-LiveTranslate via DashScope API\n\n```python\nimport os\nimport time\nimport base64\nimport asyncio\nimport json\nimport websockets\nimport pyaudio\nimport queue\nimport threading\nimport traceback\n\n\nclass LiveTranslateClient:\n \"\"\"Client for the DashScope live-translation service: captures mic audio, sends it to the server, and plays back the translated speech.\"\"\"\n\n def __init__(self, api_key: str, target_language: str = \"en\", *, audio_enabled: bool = True):\n if not api_key:\n raise ValueError(\"API key cannot be empty.\")\n\n self.api_key = api_key\n self.target_language = target_language\n self.audio_enabled = audio_enabled\n self.ws = None\n self.api_url = \"wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen3.5-livetranslate-flash-realtime\"\n\n # Audio input parameters (microphone capture)\n self.input_rate = 16000\n self.input_chunk = 1600\n self.input_format = pyaudio.paInt16\n self.input_channels = 1\n\n # Audio output parameters (local playback)\n self.output_rate = 24000\n self.output_chunk = 2400\n self.output_format = pyaudio.paInt16\n self.output_channels = 1\n\n # Runtime state and playback resources\n self.is_connected = False\n self.audio_player_thread = None\n self.audio_playback_queue = queue.Queue()\n self.pyaudio_instance = pyaudio.PyAudio()\n\n async def connect(self):\n \"\"\"Open a WebSocket connection to the translation service.\"\"\"\n headers = {\"Authorization\": f\"Bearer {self.api_key}\"}\n try:\n self.ws = await websockets.connect(self.api_url, additional_headers=headers)\n self.is_connected = True\n print(f\"Successfully connected to server: {self.api_url}\")\n await self.configure_session()\n except Exception as e:\n print(f\"Connection failed: {e}\")\n self.is_connected = False\n raise\n\n async def configure_session(self):\n \"\"\"Configure the translation session: target language, audio formats, and optional features.\"\"\"\n config = {\n \"event_id\": f\"event_{int(time.time() * 1000)}\",\n \"type\": \"session.update\",\n \"session\": {\n # `modalities` decides what the server returns:\n # [\"text\", \"audio\"] — both translated text and synthesized speech (recommended)\n # [\"text\"] — translated text only\n \"modalities\": [\"text\", \"audio\"] if self.audio_enabled else [\"text\"],\n \"input_audio_format\": \"pcm\",\n \"output_audio_format\": \"pcm\",\n # `input_audio_transcription`: enable source-language ASR.\n # Setting `model` to 'qwen3-asr-flash-realtime' also streams back the source transcript.\n # \"input_audio_transcription\": {\n # \"model\": \"qwen3-asr-flash-realtime\",\n # \"language\": \"zh\" # source language; defaults to 'en'\n # },\n \"translation\": {\n \"language\": self.target_language,\n # `corpus`: register hotwords to boost accuracy on proper nouns and domain-specific terms.\n # \"corpus\": {\n # \"phrases\": {\n # \"人工智能\": \"Artificial Intelligence\",\n # \"机器学习\": \"Machine Learning\"\n # }\n # }\n }\n }\n }\n print(f\"Sending session config: {json.dumps(config, indent=2, ensure_ascii=False)}\")\n await self.ws.send(json.dumps(config))\n\n async def send_audio_chunk(self, audio_data: bytes):\n \"\"\"Base64-encode an audio chunk and send it to the server.\"\"\"\n if not self.is_connected:\n return\n\n event = {\n \"event_id\": f\"event_{int(time.time() * 1000)}\",\n \"type\": \"input_audio_buffer.append\",\n \"audio\": base64.b64encode(audio_data).decode()\n }\n await self.ws.send(json.dumps(event))\n\n async def send_image_frame(self, image_bytes: bytes, *, event_id: str | None = None):\n \"\"\"Send an image frame to the server as visual context for translation.\"\"\"\n if not self.is_connected:\n return\n if not image_bytes:\n raise ValueError(\"image_bytes cannot be empty\")\n\n image_b64 = base64.b64encode(image_bytes).decode()\n event = {\n \"event_id\": event_id or f\"event_{int(time.time() * 1000)}\",\n \"type\": \"input_image_buffer.append\",\n \"image\": image_b64,\n }\n await self.ws.send(json.dumps(event))\n\n def _audio_player_task(self):\n \"\"\"Background thread task: drain PCM chunks from the playback queue and write them to the speaker output stream.\"\"\"\n stream = self.pyaudio_instance.open(\n format=self.output_format,\n channels=self.output_channels,\n rate=self.output_rate,\n output=True,\n frames_per_buffer=self.output_chunk,\n )\n try:\n while self.is_connected or not self.audio_playback_queue.empty():\n try:\n audio_chunk = self.audio_playback_queue.get(timeout=0.1)\n if audio_chunk is None: # sentinel: stop the playback loop\n break\n stream.write(audio_chunk)\n self.audio_playback_queue.task_done()\n except queue.Empty:\n continue\n finally:\n stream.stop_stream()\n stream.close()\n\n def start_audio_player(self):\n \"\"\"Spin up the background audio playback thread (no-op when audio output is disabled).\"\"\"\n if not self.audio_enabled:\n return\n if self.audio_player_thread is None or not self.audio_player_thread.is_alive():\n self.audio_player_thread = threading.Thread(target=self._audio_player_task, daemon=True)\n self.audio_player_thread.start()\n\n async def handle_server_messages(self, on_text_received):\n \"\"\"Continuously receive and dispatch event messages pushed by the server.\"\"\"\n try:\n async for message in self.ws:\n event = json.loads(message)\n event_type = event.get(\"type\")\n if event_type == \"response.audio.delta\" and self.audio_enabled:\n audio_b64 = event.get(\"delta\", \"\")\n if audio_b64:\n audio_data = base64.b64decode(audio_b64)\n self.audio_playback_queue.put(audio_data)\n\n elif event_type == \"response.done\":\n print(\"\\n[INFO] Response complete.\")\n usage = event.get(\"response\", {}).get(\"usage\", {})\n if usage:\n print(f\"[INFO] Token usage: {json.dumps(usage, indent=2, ensure_ascii=False)}\")\n # Receive source-language ASR results (requires input_audio_transcription.model to be enabled)\n # elif event_type == \"conversation.item.input_audio_transcription.text\":\n # stash = event.get(\"stash\", \"\") # streaming partial result, not yet finalized\n # print(f\"[Recognizing] {stash}\")\n # elif event_type == \"conversation.item.input_audio_transcription.completed\":\n # transcript = event.get(\"transcript\", \"\") # final transcript for an utterance\n # print(f\"[Source] {transcript}\")\n # In voice + text mode, the translation text arrives alongside synthesized audio under the `transcript` field\n elif event_type == \"response.audio_transcript.done\":\n print(\"\\n[INFO] Translation complete.\")\n text = event.get(\"transcript\", \"\")\n if text:\n print(f\"[INFO] Translation: {text}\")\n # In text-only mode, the translation arrives via response.text.done under the `text` field\n elif event_type == \"response.text.done\":\n print(\"\\n[INFO] Translation complete.\")\n text = event.get(\"text\", \"\")\n if text:\n print(f\"[INFO] Translation: {text}\")\n\n except websockets.exceptions.ConnectionClosed as e:\n print(f\"[WARNING] Connection closed: {e}\")\n self.is_connected = False\n except Exception as e:\n print(f\"[ERROR] Unknown error during message handling: {e}\")\n traceback.print_exc()\n self.is_connected = False\n\n async def start_microphone_streaming(self):\n \"\"\"Continuously capture microphone audio and stream it to the server in real time.\"\"\"\n stream = self.pyaudio_instance.open(\n format=self.input_format,\n channels=self.input_channels,\n rate=self.input_rate,\n input=True,\n frames_per_buffer=self.input_chunk\n )\n print(\"Microphone started, please begin speaking...\")\n try:\n while self.is_connected:\n audio_chunk = await asyncio.get_event_loop().run_in_executor(\n None, stream.read, self.input_chunk\n )\n await self.send_audio_chunk(audio_chunk)\n finally:\n stream.stop_stream()\n stream.close()\n\n async def close(self):\n \"\"\"Gracefully close the WebSocket connection and release audio resources.\"\"\"\n self.is_connected = False\n if self.ws:\n await self.ws.close()\n print(\"WebSocket connection closed.\")\n\n if self.audio_player_thread:\n self.audio_playback_queue.put(None) # signal the playback thread to exit\n self.audio_player_thread.join(timeout=1)\n print(\"Audio playback thread stopped.\")\n\n self.pyaudio_instance.terminate()\n print(\"PyAudio instance released.\")\n\n\ndef print_banner():\n print(\"=\" * 60)\n print(\" Powered by Qwen qwen3.5-livetranslate-flash-realtime\")\n print(\"=\" * 60 + \"\\n\")\n\n\ndef get_user_config():\n \"\"\"Collect runtime parameters from the user via CLI: output mode and target language.\"\"\"\n print(\"Select mode:\")\n print(\"1. Voice + Text [default] | 2. Text only\")\n mode_choice = input(\"Enter option (press Enter for Voice + Text): \").strip()\n audio_enabled = (mode_choice != \"2\")\n\n if audio_enabled:\n lang_map = {\n \"1\": \"en\", \"2\": \"zh\", \"3\": \"ru\", \"4\": \"fr\", \"5\": \"de\", \"6\": \"pt\",\n \"7\": \"es\", \"8\": \"it\", \"9\": \"ko\", \"10\": \"ja\", \"11\": \"yue\"\n }\n print(\"Select target translation language (Voice + Text mode):\")\n print(\"1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Korean | 10. Japanese | 11. Cantonese\")\n else:\n lang_map = {\n \"1\": \"en\", \"2\": \"zh\", \"3\": \"ru\", \"4\": \"fr\", \"5\": \"de\", \"6\": \"pt\", \"7\": \"es\", \"8\": \"it\",\n \"9\": \"id\", \"10\": \"ko\", \"11\": \"ja\", \"12\": \"vi\", \"13\": \"th\", \"14\": \"ar\",\n \"15\": \"yue\", \"16\": \"hi\", \"17\": \"el\", \"18\": \"tr\"\n }\n print(\"Select target translation language (Text only mode):\")\n print(\"1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Indonesian | 10. Korean | 11. Japanese | 12. Vietnamese | 13. Thai | 14. Arabic | 15. Cantonese | 16. Hindi | 17. Greek | 18. Turkish\")\n\n choice = input(\"Enter option (default is the first one): \").strip()\n target_language = lang_map.get(choice, next(iter(lang_map.values())))\n\n return target_language, audio_enabled\n\n\nasync def main():\n \"\"\"Program entry point: connect, configure the session, and drive the live-translation loop.\"\"\"\n print_banner()\n\n api_key = os.environ.get(\"DASHSCOPE_API_KEY\")\n if not api_key:\n print(\"[ERROR] Please set the environment variable DASHSCOPE_API_KEY\")\n print(\" Example: export DASHSCOPE_API_KEY='your_api_key_here'\")\n return\n\n target_language, audio_enabled = get_user_config()\n print(\"\\nConfiguration complete:\")\n print(f\" - Target language: {target_language}\")\n if not audio_enabled:\n print(\" - Output mode: Text only\")\n\n client = LiveTranslateClient(api_key=api_key, target_language=target_language, audio_enabled=audio_enabled)\n\n # Callback fired as translated text arrives — stream it to stdout, character by character\n def on_translation_text(text):\n print(text, end=\"\", flush=True)\n\n try:\n print(\"Connecting to the translation service...\")\n await client.connect()\n\n # Launch the audio playback thread (only does real work when audio output is enabled)\n client.start_audio_player()\n\n print(\"\\n\" + \"-\" * 60)\n print(\"Connected! Please speak into the microphone.\")\n print(\"The program will translate your speech in real time and play the results. Press Ctrl+C to exit.\")\n print(\"-\" * 60 + \"\\n\")\n\n # Run two coroutines concurrently: server-message handling + microphone audio upload\n message_handler = asyncio.create_task(client.handle_server_messages(on_translation_text))\n tasks = [message_handler]\n # Microphone capture is the translation input source — required regardless of output mode\n microphone_streamer = asyncio.create_task(client.start_microphone_streaming())\n tasks.append(microphone_streamer)\n\n await asyncio.gather(*tasks)\n\n except KeyboardInterrupt:\n print(\"\\n\\nUser interrupted, exiting...\")\n except Exception as e:\n print(f\"\\nFatal error occurred: {e}\")\n finally:\n print(\"\\nCleaning up resources...\")\n await client.close()\n print(\"Program exited.\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n## Future Directions\n\nWe will continue exploring the capability boundaries of multimodal translation and focus on the following directions:\n\n- **Lower latency**: keep reducing end-to-end simultaneous interpretation latency toward real-time experience limits.\n- **More languages and dialects**: expand input/output coverage for low-resource languages, regional dialects, and cross-regional expressions.\n- **Longer context and stronger consistency**: maintain terminology, names, and context consistency in long meetings and multi-turn dialogues.\n- **Higher-fidelity voice cloning**: preserve speaker characteristics while restoring ambient sounds and on-site atmosphere more naturally.\n- **Richer interaction modes**: support multilingual, mixed-dialect expression, speaker separation, and joint multimodal modeling with gestures, lip movement, and expressions.\n\n## Citation\n\nFeel free to cite the following article if you find Qwen3.5-LiveTranslate helpful:\n\n```bibtex\n@misc{qwen35livetranslateblog,\n title = {Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right},\n url = {https://qwen.ai/blog?id=qwen3.5-livetranslate},\n author = {Qwen Team},\n month = {May},\n year = {2026}\n}\n```\n", "authors": [ "QwenTeam" ], "captured_at": "2026-10-19T01:26:53.998049939+00:00", "category": "ai", "content_html": "

\"Qwen3.5-LiveTranslate

\n

DashScope Demo

\n

Qwen3.5-LiveTranslate-Flash is the latest simultaneous interpretation model in the Qwen family, built on top of Qwen3.5-Omni. It delivers real-time, multimodal translation that not only hears and translates speech, but also sees and understands visual context to produce more accurate translations. Compared with its predecessor Qwen3-LiveTranslate, Qwen3.5-LiveTranslate-Flash brings major upgrades across language coverage, latency, voice cloning, and terminology handling, making it well-suited for international meetings, livestream localization, online classrooms, and business negotiations.

\n

Key Highlights

\n\n

Performance

\n

We evaluate Qwen3.5-LiveTranslate-Flash in both offline and real-time (streaming) settings.

\n

Offline Translation

\n

On public multilingual speech translation benchmarks (FLEURS, CoVoST2), Qwen3.5-LiveTranslate-Flash achieves higher translation accuracy than mainstream commerical large speech models, significantly surpasses its predecessor Qwen3-LiveTranslate-Flash, and delivers breakthroughs in both language coverage and translation quality.

\n

Overview English → X Next

\n

\"image\"

\n

Overview X → English Next

\n

\"image\"

\n

FLEURS English → X Next

\n

\"image\"

\n

FLEURS X → English Next

\n

\"image\"

\n

CoVoST2 English → X Next

\n

\"image\"

\n

CoVoST2 X → English Next

\n

\"image\"

\n

Real-Time Translation

\n

With the Readable Unit streaming strategy, Qwen3.5-LiveTranslate-Flash reduces first-token latency by 3.45 s and per-token latency by 1.88 s compared to Qwen3-LiveTranslate-Flash, achieving an average speech-to-speech per-token latency of 2.8 s, with virtually no loss in translation quality.

\n

Overview Next

\n

\"image\"

\n

Model Architecture

\n

Qwen3.5-LiveTranslate is a translation large model built on the Qwen3.5-Omni Thinker-Talker architecture. The Thinker receives interleaved visual and audio inputs and generates text translations, while the Talker takes the translated text and source audio to produce speech with crosslingual voice cloning. For real-time simultaneous interpretation, we adopt a chunk-wise streaming input mechanism and introduce Readable Unit tags to control speech synthesis granularity, effectively reducing interpretation latency. Meanwhile, dynamic crosslingual voice cloning enables the model to preserve the speaker’s original vocal characteristics during real-time translation.

\n

\"Qwen3.5-LiveTranslate

\n

Qwen3.5-LiveTranslate model architecture overview

\n

More Supported Languages

\n

Compared to Qwen3-LiveTranslate, Qwen3.5-LiveTranslate significantly expands language coverage. The support of input audio and output text grows from 18 to 60 languages, and output audio support from 10 to 29 languages, enabling a far wider range of cross-lingual translation combinations across global scenarios.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Qwen3-LiveTranslateQwen3.5-LiveTranslate
Input ModalityAudio / VideoAudio / Video
Inference ModeOffline / StreamingOffline / Streaming
Voice Cloning✗✓ (3 modes: pre-registered / clone-once / real-time)
HotwordsUp to 1,000Up to 1,000
Input Audio Languages & Output Text Languages18 languages Chinese, English, Russian, French, German, Portuguese, Spanish, Italian, Indonesian, Korean, Japanese, Vietnamese, Thai, Arabic, Cantonese, Hindi, Greek, Turkish60 languages Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Nynorsk, Odia, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, Vietnamese
Output Audio Languages10 languages Chinese, English, French, German, Russian, Italian, Spanish, Portuguese, Japanese, Korean29 languages Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Filipino, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, Persian
\n

🎬 See It in Action

\n

International Meeting

\n

A multilingual business meeting where participants speak in different languages and switch between them mid-sentence. Qwen3.5-LiveTranslate handles code-switching, diverse accents, and domain-specific terminology in real time — delivering fluent, natural translations without missing a beat.

\n

Traveling Abroad

\n

A real-world travel scenario powered by Qwen AI Glasses: a Chinese tourist orders food at a local restaurant in Thailand. The model performs live Thai-to-Chinese translation on-device, combining visual context from the menu with spoken dialogue to produce accurate, context-aware translations — making cross-language communication effortless on the go.

\n

Livestream Scenarios

\n

E-commerce livestream translation scenario. Qwen3.5-LiveTranslate accurately translates product specifications and numerical information, ensuring precise cross-language delivery of product parameters.

\n

Classical Chinese Translation

\n

A scene from Romance of the Three Kingdoms narrated in classical Chinese (文言文). Qwen3.5-LiveTranslate accurately interprets and translates archaic Chinese prose into modern English, demonstrating its ability to handle literary and historical language beyond everyday speech.

\n

Visual Disambiguation

\n

Qwen3.5-LiveTranslate leverages visual context to resolve translation ambiguities. When a word or phrase has multiple possible meanings, the model uses what it sees — on-screen text, objects, or scene context — to select the correct interpretation, producing translations that are both accurate and contextually grounded.

\n

Using Qwen3.5-LiveTranslate via DashScope API

\n
import os\nimport time\nimport base64\nimport asyncio\nimport json\nimport websockets\nimport pyaudio\nimport queue\nimport threading\nimport traceback\n\n\nclass LiveTranslateClient:\n    """Client for the DashScope live-translation service: captures mic audio, sends it to the server, and plays back the translated speech."""\n\n    def __init__(self, api_key: str, target_language: str = "en", *, audio_enabled: bool = True):\n        if not api_key:\n            raise ValueError("API key cannot be empty.")\n\n        self.api_key = api_key\n        self.target_language = target_language\n        self.audio_enabled = audio_enabled\n        self.ws = None\n        self.api_url = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen3.5-livetranslate-flash-realtime"\n\n        # Audio input parameters (microphone capture)\n        self.input_rate = 16000\n        self.input_chunk = 1600\n        self.input_format = pyaudio.paInt16\n        self.input_channels = 1\n\n        # Audio output parameters (local playback)\n        self.output_rate = 24000\n        self.output_chunk = 2400\n        self.output_format = pyaudio.paInt16\n        self.output_channels = 1\n\n        # Runtime state and playback resources\n        self.is_connected = False\n        self.audio_player_thread = None\n        self.audio_playback_queue = queue.Queue()\n        self.pyaudio_instance = pyaudio.PyAudio()\n\n    async def connect(self):\n        """Open a WebSocket connection to the translation service."""\n        headers = {"Authorization": f"Bearer {self.api_key}"}\n        try:\n            self.ws = await websockets.connect(self.api_url, additional_headers=headers)\n            self.is_connected = True\n            print(f"Successfully connected to server: {self.api_url}")\n            await self.configure_session()\n        except Exception as e:\n            print(f"Connection failed: {e}")\n            self.is_connected = False\n            raise\n\n    async def configure_session(self):\n        """Configure the translation session: target language, audio formats, and optional features."""\n        config = {\n            "event_id": f"event_{int(time.time() * 1000)}",\n            "type": "session.update",\n            "session": {\n                # `modalities` decides what the server returns:\n                #   ["text", "audio"] — both translated text and synthesized speech (recommended)\n                #   ["text"]          — translated text only\n                "modalities": ["text", "audio"] if self.audio_enabled else ["text"],\n                "input_audio_format": "pcm",\n                "output_audio_format": "pcm",\n                # `input_audio_transcription`: enable source-language ASR.\n                # Setting `model` to 'qwen3-asr-flash-realtime' also streams back the source transcript.\n                # "input_audio_transcription": {\n                #     "model": "qwen3-asr-flash-realtime",\n                #     "language": "zh"  # source language; defaults to 'en'\n                # },\n                "translation": {\n                    "language": self.target_language,\n                    # `corpus`: register hotwords to boost accuracy on proper nouns and domain-specific terms.\n                    # "corpus": {\n                    #     "phrases": {\n                    #         "人工智能": "Artificial Intelligence",\n                    #         "机器学习": "Machine Learning"\n                    #     }\n                    # }\n                }\n            }\n        }\n        print(f"Sending session config: {json.dumps(config, indent=2, ensure_ascii=False)}")\n        await self.ws.send(json.dumps(config))\n\n    async def send_audio_chunk(self, audio_data: bytes):\n        """Base64-encode an audio chunk and send it to the server."""\n        if not self.is_connected:\n            return\n\n        event = {\n            "event_id": f"event_{int(time.time() * 1000)}",\n            "type": "input_audio_buffer.append",\n            "audio": base64.b64encode(audio_data).decode()\n        }\n        await self.ws.send(json.dumps(event))\n\n    async def send_image_frame(self, image_bytes: bytes, *, event_id: str | None = None):\n        """Send an image frame to the server as visual context for translation."""\n        if not self.is_connected:\n            return\n        if not image_bytes:\n            raise ValueError("image_bytes cannot be empty")\n\n        image_b64 = base64.b64encode(image_bytes).decode()\n        event = {\n            "event_id": event_id or f"event_{int(time.time() * 1000)}",\n            "type": "input_image_buffer.append",\n            "image": image_b64,\n        }\n        await self.ws.send(json.dumps(event))\n\n    def _audio_player_task(self):\n        """Background thread task: drain PCM chunks from the playback queue and write them to the speaker output stream."""\n        stream = self.pyaudio_instance.open(\n            format=self.output_format,\n            channels=self.output_channels,\n            rate=self.output_rate,\n            output=True,\n            frames_per_buffer=self.output_chunk,\n        )\n        try:\n            while self.is_connected or not self.audio_playback_queue.empty():\n                try:\n                    audio_chunk = self.audio_playback_queue.get(timeout=0.1)\n                    if audio_chunk is None:  # sentinel: stop the playback loop\n                        break\n                    stream.write(audio_chunk)\n                    self.audio_playback_queue.task_done()\n                except queue.Empty:\n                    continue\n        finally:\n            stream.stop_stream()\n            stream.close()\n\n    def start_audio_player(self):\n        """Spin up the background audio playback thread (no-op when audio output is disabled)."""\n        if not self.audio_enabled:\n            return\n        if self.audio_player_thread is None or not self.audio_player_thread.is_alive():\n            self.audio_player_thread = threading.Thread(target=self._audio_player_task, daemon=True)\n            self.audio_player_thread.start()\n\n    async def handle_server_messages(self, on_text_received):\n        """Continuously receive and dispatch event messages pushed by the server."""\n        try:\n            async for message in self.ws:\n                event = json.loads(message)\n                event_type = event.get("type")\n                if event_type == "response.audio.delta" and self.audio_enabled:\n                    audio_b64 = event.get("delta", "")\n                    if audio_b64:\n                        audio_data = base64.b64decode(audio_b64)\n                        self.audio_playback_queue.put(audio_data)\n\n                elif event_type == "response.done":\n                    print("\\n[INFO] Response complete.")\n                    usage = event.get("response", {}).get("usage", {})\n                    if usage:\n                        print(f"[INFO] Token usage: {json.dumps(usage, indent=2, ensure_ascii=False)}")\n                # Receive source-language ASR results (requires input_audio_transcription.model to be enabled)\n                # elif event_type == "conversation.item.input_audio_transcription.text":\n                #     stash = event.get("stash", "")  # streaming partial result, not yet finalized\n                #     print(f"[Recognizing] {stash}")\n                # elif event_type == "conversation.item.input_audio_transcription.completed":\n                #     transcript = event.get("transcript", "")  # final transcript for an utterance\n                #     print(f"[Source] {transcript}")\n                # In voice + text mode, the translation text arrives alongside synthesized audio under the `transcript` field\n                elif event_type == "response.audio_transcript.done":\n                    print("\\n[INFO] Translation complete.")\n                    text = event.get("transcript", "")\n                    if text:\n                        print(f"[INFO] Translation: {text}")\n                # In text-only mode, the translation arrives via response.text.done under the `text` field\n                elif event_type == "response.text.done":\n                    print("\\n[INFO] Translation complete.")\n                    text = event.get("text", "")\n                    if text:\n                        print(f"[INFO] Translation: {text}")\n\n        except websockets.exceptions.ConnectionClosed as e:\n            print(f"[WARNING] Connection closed: {e}")\n            self.is_connected = False\n        except Exception as e:\n            print(f"[ERROR] Unknown error during message handling: {e}")\n            traceback.print_exc()\n            self.is_connected = False\n\n    async def start_microphone_streaming(self):\n        """Continuously capture microphone audio and stream it to the server in real time."""\n        stream = self.pyaudio_instance.open(\n            format=self.input_format,\n            channels=self.input_channels,\n            rate=self.input_rate,\n            input=True,\n            frames_per_buffer=self.input_chunk\n        )\n        print("Microphone started, please begin speaking...")\n        try:\n            while self.is_connected:\n                audio_chunk = await asyncio.get_event_loop().run_in_executor(\n                    None, stream.read, self.input_chunk\n                )\n                await self.send_audio_chunk(audio_chunk)\n        finally:\n            stream.stop_stream()\n            stream.close()\n\n    async def close(self):\n        """Gracefully close the WebSocket connection and release audio resources."""\n        self.is_connected = False\n        if self.ws:\n            await self.ws.close()\n            print("WebSocket connection closed.")\n\n        if self.audio_player_thread:\n            self.audio_playback_queue.put(None)  # signal the playback thread to exit\n            self.audio_player_thread.join(timeout=1)\n            print("Audio playback thread stopped.")\n\n        self.pyaudio_instance.terminate()\n        print("PyAudio instance released.")\n\n\ndef print_banner():\n    print("=" * 60)\n    print("  Powered by Qwen qwen3.5-livetranslate-flash-realtime")\n    print("=" * 60 + "\\n")\n\n\ndef get_user_config():\n    """Collect runtime parameters from the user via CLI: output mode and target language."""\n    print("Select mode:")\n    print("1. Voice + Text [default] | 2. Text only")\n    mode_choice = input("Enter option (press Enter for Voice + Text): ").strip()\n    audio_enabled = (mode_choice != "2")\n\n    if audio_enabled:\n        lang_map = {\n            "1": "en", "2": "zh", "3": "ru", "4": "fr", "5": "de", "6": "pt",\n            "7": "es", "8": "it", "9": "ko", "10": "ja", "11": "yue"\n        }\n        print("Select target translation language (Voice + Text mode):")\n        print("1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Korean | 10. Japanese | 11. Cantonese")\n    else:\n        lang_map = {\n            "1": "en", "2": "zh", "3": "ru", "4": "fr", "5": "de", "6": "pt", "7": "es", "8": "it",\n            "9": "id", "10": "ko", "11": "ja", "12": "vi", "13": "th", "14": "ar",\n            "15": "yue", "16": "hi", "17": "el", "18": "tr"\n        }\n        print("Select target translation language (Text only mode):")\n        print("1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Indonesian | 10. Korean | 11. Japanese | 12. Vietnamese | 13. Thai | 14. Arabic | 15. Cantonese | 16. Hindi | 17. Greek | 18. Turkish")\n\n    choice = input("Enter option (default is the first one): ").strip()\n    target_language = lang_map.get(choice, next(iter(lang_map.values())))\n\n    return target_language, audio_enabled\n\n\nasync def main():\n    """Program entry point: connect, configure the session, and drive the live-translation loop."""\n    print_banner()\n\n    api_key = os.environ.get("DASHSCOPE_API_KEY")\n    if not api_key:\n        print("[ERROR] Please set the environment variable DASHSCOPE_API_KEY")\n        print("  Example: export DASHSCOPE_API_KEY='your_api_key_here'")\n        return\n\n    target_language, audio_enabled = get_user_config()\n    print("\\nConfiguration complete:")\n    print(f"  - Target language: {target_language}")\n    if not audio_enabled:\n        print("  - Output mode: Text only")\n\n    client = LiveTranslateClient(api_key=api_key, target_language=target_language, audio_enabled=audio_enabled)\n\n    # Callback fired as translated text arrives — stream it to stdout, character by character\n    def on_translation_text(text):\n        print(text, end="", flush=True)\n\n    try:\n        print("Connecting to the translation service...")\n        await client.connect()\n\n        # Launch the audio playback thread (only does real work when audio output is enabled)\n        client.start_audio_player()\n\n        print("\\n" + "-" * 60)\n        print("Connected! Please speak into the microphone.")\n        print("The program will translate your speech in real time and play the results. Press Ctrl+C to exit.")\n        print("-" * 60 + "\\n")\n\n        # Run two coroutines concurrently: server-message handling + microphone audio upload\n        message_handler = asyncio.create_task(client.handle_server_messages(on_translation_text))\n        tasks = [message_handler]\n        # Microphone capture is the translation input source — required regardless of output mode\n        microphone_streamer = asyncio.create_task(client.start_microphone_streaming())\n        tasks.append(microphone_streamer)\n\n        await asyncio.gather(*tasks)\n\n    except KeyboardInterrupt:\n        print("\\n\\nUser interrupted, exiting...")\n    except Exception as e:\n        print(f"\\nFatal error occurred: {e}")\n    finally:\n        print("\\nCleaning up resources...")\n        await client.close()\n        print("Program exited.")\n\n\nif __name__ == "__main__":\n    asyncio.run(main())\n
\n

Future Directions

\n

We will continue exploring the capability boundaries of multimodal translation and focus on the following directions:

\n\n

Citation

\n

Feel free to cite the following article if you find Qwen3.5-LiveTranslate helpful:

\n
@misc{qwen35livetranslateblog,\n    title = {Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right},\n    url = {https://qwen.ai/blog?id=qwen3.5-livetranslate},\n    author = {Qwen Team},\n    month = {May},\n    year = {2026}\n}\n
\n", "content_markdown": "![Qwen3.5-LiveTranslate Main Image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/main.png)\n\n[DashScope](https://www.alibabacloud.com/help/en/model-studio/qwen3-5-livetranslate-flash-realtime) [Demo](https://omni.qwen.ai/live-translate)\n\n**Qwen3.5-LiveTranslate-Flash** is the latest simultaneous interpretation model in the Qwen family, built on top of Qwen3.5-Omni. It delivers real-time, multimodal translation that not only hears and translates speech, but also sees and understands visual context to produce more accurate translations. Compared with its predecessor Qwen3-LiveTranslate, Qwen3.5-LiveTranslate-Flash brings major upgrades across language coverage, latency, voice cloning, and terminology handling, making it well-suited for international meetings, livestream localization, online classrooms, and business negotiations.\n\n### Key Highlights\n\n- **Massively expanded language coverage: understands 18 → 60 languages, speaks 10 → 29 languages.** The language support of input audio and output text has grown from 18 to 60, and output audio language support from 10 to 29, covering far more cross-lingual combinations to meet multilingual interpretation needs in international meetings, livestream localization, online classrooms, and business negotiations.\n\n- **Ultra-low latency: powered by Readable Unit technology, faster text and speech output.** A novel Readable Unit real-time translation technique achieves more aggressive streaming output while preserving translation readability and semantic consistency. Average speech-to-speech per-token latency is reduced to to 2.8 seconds, ideal for latency-sensitive scenarios such as livestreams, co-hosting, and press conferences.\n\n- **Real-time voice cloning: one sentence to start, instantly “interpret in your voice”.** During simultaneous interpretation, the system automatically replicates the speaker’s vocal characteristics, keeping the translated speech sounding like “the same person” across languages, enhancing immersion and identity consistency, especially critical for streamers, guests, and hosts.\n\n- **Hotword enhancement: proper nouns and industry terms “recognized right, written right, translated right”.** Built-in Hotword capability prioritizes the recognition and translation of names, places, brand names, product models, and industry terminology. Hotwords can be dynamically configured and updated in real time per scenario, significantly reducing terminology mistranslation risk, well-suited for technical launches, medical/legal/financial meetings, and enterprise training.\n\n## Performance\n\nWe evaluate Qwen3.5-LiveTranslate-Flash in both offline and real-time (streaming) settings.\n\n### Offline Translation\n\nOn public multilingual speech translation benchmarks (FLEURS, CoVoST2), Qwen3.5-LiveTranslate-Flash achieves higher translation accuracy than mainstream commerical large speech models, significantly surpasses its predecessor Qwen3-LiveTranslate-Flash, and delivers breakthroughs in both language coverage and translation quality.\n\nOverview English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_overview_en-xx.png)\n\nOverview X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_overview_xx-en.png)\n\nFLEURS English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_fleurs_en-xx.png)\n\nFLEURS X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_fleurs_xx-en.png)\n\nCoVoST2 English → X Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_covost_en-xx.png)\n\nCoVoST2 X → English Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/offline_covost_xx-en.png)\n\n### Real-Time Translation\n\nWith the Readable Unit streaming strategy, Qwen3.5-LiveTranslate-Flash reduces first-token latency by **3.45 s** and per-token latency by **1.88 s** compared to Qwen3-LiveTranslate-Flash, achieving an average speech-to-speech per-token latency of **2.8 s**, with virtually no loss in translation quality.\n\nOverview Next\n\n![image](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/online_overview.png)\n\n## Model Architecture\n\nQwen3.5-LiveTranslate is a translation large model built on the Qwen3.5-Omni Thinker-Talker architecture. The Thinker receives interleaved visual and audio inputs and generates text translations, while the Talker takes the translated text and source audio to produce speech with crosslingual voice cloning. For real-time simultaneous interpretation, we adopt a chunk-wise streaming input mechanism and introduce Readable Unit tags to control speech synthesis granularity, effectively reducing interpretation latency. Meanwhile, dynamic crosslingual voice cloning enables the model to preserve the speaker’s original vocal characteristics during real-time translation.\n\n![Qwen3.5-LiveTranslate model architecture overview](https://qianwen-res.oss-accelerate.aliyuncs.com/Qwen3.5-LiveTranslate/model_arch_v3.png)\n\nQwen3.5-LiveTranslate model architecture overview\n\n## More Supported Languages\n\nCompared to Qwen3-LiveTranslate, Qwen3.5-LiveTranslate significantly expands language coverage. The support of input audio and output text grows from 18 to 60 languages, and output audio support from 10 to 29 languages, enabling a far wider range of cross-lingual translation combinations across global scenarios.\n\n| | Qwen3-LiveTranslate | Qwen3.5-LiveTranslate |\n| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Input Modality | Audio / Video | Audio / Video |\n| Inference Mode | Offline / Streaming | Offline / Streaming |\n| Voice Cloning | ✗ | ✓ (3 modes: pre-registered / clone-once / real-time) |\n| Hotwords | Up to 1,000 | Up to 1,000 |\n| Input Audio Languages & Output Text Languages | **18 languages** Chinese, English, Russian, French, German, Portuguese, Spanish, Italian, Indonesian, Korean, Japanese, Vietnamese, Thai, Arabic, Cantonese, Hindi, Greek, Turkish | **60 languages** Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Nynorsk, Odia, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, Vietnamese |\n| Output Audio Languages | **10 languages** Chinese, English, French, German, Russian, Italian, Spanish, Portuguese, Japanese, Korean | **29 languages** Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Filipino, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, Persian |\n\n## 🎬 See It in Action\n\n### International Meeting\n\nA multilingual business meeting where participants speak in different languages and switch between them mid-sentence. Qwen3.5-LiveTranslate handles code-switching, diverse accents, and domain-specific terminology in real time — delivering fluent, natural translations without missing a beat.\n\n### Traveling Abroad\n\nA real-world travel scenario powered by Qwen AI Glasses: a Chinese tourist orders food at a local restaurant in Thailand. The model performs live Thai-to-Chinese translation on-device, combining visual context from the menu with spoken dialogue to produce accurate, context-aware translations — making cross-language communication effortless on the go.\n\n### Livestream Scenarios\n\nE-commerce livestream translation scenario. Qwen3.5-LiveTranslate accurately translates product specifications and numerical information, ensuring precise cross-language delivery of product parameters.\n\n### Classical Chinese Translation\n\nA scene from *Romance of the Three Kingdoms* narrated in classical Chinese (文言文). Qwen3.5-LiveTranslate accurately interprets and translates archaic Chinese prose into modern English, demonstrating its ability to handle literary and historical language beyond everyday speech.\n\n### Visual Disambiguation\n\nQwen3.5-LiveTranslate leverages visual context to resolve translation ambiguities. When a word or phrase has multiple possible meanings, the model uses what it sees — on-screen text, objects, or scene context — to select the correct interpretation, producing translations that are both accurate and contextually grounded.\n\n## Using Qwen3.5-LiveTranslate via DashScope API\n\n```python\nimport os\nimport time\nimport base64\nimport asyncio\nimport json\nimport websockets\nimport pyaudio\nimport queue\nimport threading\nimport traceback\n\n\nclass LiveTranslateClient:\n \"\"\"Client for the DashScope live-translation service: captures mic audio, sends it to the server, and plays back the translated speech.\"\"\"\n\n def __init__(self, api_key: str, target_language: str = \"en\", *, audio_enabled: bool = True):\n if not api_key:\n raise ValueError(\"API key cannot be empty.\")\n\n self.api_key = api_key\n self.target_language = target_language\n self.audio_enabled = audio_enabled\n self.ws = None\n self.api_url = \"wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen3.5-livetranslate-flash-realtime\"\n\n # Audio input parameters (microphone capture)\n self.input_rate = 16000\n self.input_chunk = 1600\n self.input_format = pyaudio.paInt16\n self.input_channels = 1\n\n # Audio output parameters (local playback)\n self.output_rate = 24000\n self.output_chunk = 2400\n self.output_format = pyaudio.paInt16\n self.output_channels = 1\n\n # Runtime state and playback resources\n self.is_connected = False\n self.audio_player_thread = None\n self.audio_playback_queue = queue.Queue()\n self.pyaudio_instance = pyaudio.PyAudio()\n\n async def connect(self):\n \"\"\"Open a WebSocket connection to the translation service.\"\"\"\n headers = {\"Authorization\": f\"Bearer {self.api_key}\"}\n try:\n self.ws = await websockets.connect(self.api_url, additional_headers=headers)\n self.is_connected = True\n print(f\"Successfully connected to server: {self.api_url}\")\n await self.configure_session()\n except Exception as e:\n print(f\"Connection failed: {e}\")\n self.is_connected = False\n raise\n\n async def configure_session(self):\n \"\"\"Configure the translation session: target language, audio formats, and optional features.\"\"\"\n config = {\n \"event_id\": f\"event_{int(time.time() * 1000)}\",\n \"type\": \"session.update\",\n \"session\": {\n # `modalities` decides what the server returns:\n # [\"text\", \"audio\"] — both translated text and synthesized speech (recommended)\n # [\"text\"] — translated text only\n \"modalities\": [\"text\", \"audio\"] if self.audio_enabled else [\"text\"],\n \"input_audio_format\": \"pcm\",\n \"output_audio_format\": \"pcm\",\n # `input_audio_transcription`: enable source-language ASR.\n # Setting `model` to 'qwen3-asr-flash-realtime' also streams back the source transcript.\n # \"input_audio_transcription\": {\n # \"model\": \"qwen3-asr-flash-realtime\",\n # \"language\": \"zh\" # source language; defaults to 'en'\n # },\n \"translation\": {\n \"language\": self.target_language,\n # `corpus`: register hotwords to boost accuracy on proper nouns and domain-specific terms.\n # \"corpus\": {\n # \"phrases\": {\n # \"人工智能\": \"Artificial Intelligence\",\n # \"机器学习\": \"Machine Learning\"\n # }\n # }\n }\n }\n }\n print(f\"Sending session config: {json.dumps(config, indent=2, ensure_ascii=False)}\")\n await self.ws.send(json.dumps(config))\n\n async def send_audio_chunk(self, audio_data: bytes):\n \"\"\"Base64-encode an audio chunk and send it to the server.\"\"\"\n if not self.is_connected:\n return\n\n event = {\n \"event_id\": f\"event_{int(time.time() * 1000)}\",\n \"type\": \"input_audio_buffer.append\",\n \"audio\": base64.b64encode(audio_data).decode()\n }\n await self.ws.send(json.dumps(event))\n\n async def send_image_frame(self, image_bytes: bytes, *, event_id: str | None = None):\n \"\"\"Send an image frame to the server as visual context for translation.\"\"\"\n if not self.is_connected:\n return\n if not image_bytes:\n raise ValueError(\"image_bytes cannot be empty\")\n\n image_b64 = base64.b64encode(image_bytes).decode()\n event = {\n \"event_id\": event_id or f\"event_{int(time.time() * 1000)}\",\n \"type\": \"input_image_buffer.append\",\n \"image\": image_b64,\n }\n await self.ws.send(json.dumps(event))\n\n def _audio_player_task(self):\n \"\"\"Background thread task: drain PCM chunks from the playback queue and write them to the speaker output stream.\"\"\"\n stream = self.pyaudio_instance.open(\n format=self.output_format,\n channels=self.output_channels,\n rate=self.output_rate,\n output=True,\n frames_per_buffer=self.output_chunk,\n )\n try:\n while self.is_connected or not self.audio_playback_queue.empty():\n try:\n audio_chunk = self.audio_playback_queue.get(timeout=0.1)\n if audio_chunk is None: # sentinel: stop the playback loop\n break\n stream.write(audio_chunk)\n self.audio_playback_queue.task_done()\n except queue.Empty:\n continue\n finally:\n stream.stop_stream()\n stream.close()\n\n def start_audio_player(self):\n \"\"\"Spin up the background audio playback thread (no-op when audio output is disabled).\"\"\"\n if not self.audio_enabled:\n return\n if self.audio_player_thread is None or not self.audio_player_thread.is_alive():\n self.audio_player_thread = threading.Thread(target=self._audio_player_task, daemon=True)\n self.audio_player_thread.start()\n\n async def handle_server_messages(self, on_text_received):\n \"\"\"Continuously receive and dispatch event messages pushed by the server.\"\"\"\n try:\n async for message in self.ws:\n event = json.loads(message)\n event_type = event.get(\"type\")\n if event_type == \"response.audio.delta\" and self.audio_enabled:\n audio_b64 = event.get(\"delta\", \"\")\n if audio_b64:\n audio_data = base64.b64decode(audio_b64)\n self.audio_playback_queue.put(audio_data)\n\n elif event_type == \"response.done\":\n print(\"\\n[INFO] Response complete.\")\n usage = event.get(\"response\", {}).get(\"usage\", {})\n if usage:\n print(f\"[INFO] Token usage: {json.dumps(usage, indent=2, ensure_ascii=False)}\")\n # Receive source-language ASR results (requires input_audio_transcription.model to be enabled)\n # elif event_type == \"conversation.item.input_audio_transcription.text\":\n # stash = event.get(\"stash\", \"\") # streaming partial result, not yet finalized\n # print(f\"[Recognizing] {stash}\")\n # elif event_type == \"conversation.item.input_audio_transcription.completed\":\n # transcript = event.get(\"transcript\", \"\") # final transcript for an utterance\n # print(f\"[Source] {transcript}\")\n # In voice + text mode, the translation text arrives alongside synthesized audio under the `transcript` field\n elif event_type == \"response.audio_transcript.done\":\n print(\"\\n[INFO] Translation complete.\")\n text = event.get(\"transcript\", \"\")\n if text:\n print(f\"[INFO] Translation: {text}\")\n # In text-only mode, the translation arrives via response.text.done under the `text` field\n elif event_type == \"response.text.done\":\n print(\"\\n[INFO] Translation complete.\")\n text = event.get(\"text\", \"\")\n if text:\n print(f\"[INFO] Translation: {text}\")\n\n except websockets.exceptions.ConnectionClosed as e:\n print(f\"[WARNING] Connection closed: {e}\")\n self.is_connected = False\n except Exception as e:\n print(f\"[ERROR] Unknown error during message handling: {e}\")\n traceback.print_exc()\n self.is_connected = False\n\n async def start_microphone_streaming(self):\n \"\"\"Continuously capture microphone audio and stream it to the server in real time.\"\"\"\n stream = self.pyaudio_instance.open(\n format=self.input_format,\n channels=self.input_channels,\n rate=self.input_rate,\n input=True,\n frames_per_buffer=self.input_chunk\n )\n print(\"Microphone started, please begin speaking...\")\n try:\n while self.is_connected:\n audio_chunk = await asyncio.get_event_loop().run_in_executor(\n None, stream.read, self.input_chunk\n )\n await self.send_audio_chunk(audio_chunk)\n finally:\n stream.stop_stream()\n stream.close()\n\n async def close(self):\n \"\"\"Gracefully close the WebSocket connection and release audio resources.\"\"\"\n self.is_connected = False\n if self.ws:\n await self.ws.close()\n print(\"WebSocket connection closed.\")\n\n if self.audio_player_thread:\n self.audio_playback_queue.put(None) # signal the playback thread to exit\n self.audio_player_thread.join(timeout=1)\n print(\"Audio playback thread stopped.\")\n\n self.pyaudio_instance.terminate()\n print(\"PyAudio instance released.\")\n\n\ndef print_banner():\n print(\"=\" * 60)\n print(\" Powered by Qwen qwen3.5-livetranslate-flash-realtime\")\n print(\"=\" * 60 + \"\\n\")\n\n\ndef get_user_config():\n \"\"\"Collect runtime parameters from the user via CLI: output mode and target language.\"\"\"\n print(\"Select mode:\")\n print(\"1. Voice + Text [default] | 2. Text only\")\n mode_choice = input(\"Enter option (press Enter for Voice + Text): \").strip()\n audio_enabled = (mode_choice != \"2\")\n\n if audio_enabled:\n lang_map = {\n \"1\": \"en\", \"2\": \"zh\", \"3\": \"ru\", \"4\": \"fr\", \"5\": \"de\", \"6\": \"pt\",\n \"7\": \"es\", \"8\": \"it\", \"9\": \"ko\", \"10\": \"ja\", \"11\": \"yue\"\n }\n print(\"Select target translation language (Voice + Text mode):\")\n print(\"1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Korean | 10. Japanese | 11. Cantonese\")\n else:\n lang_map = {\n \"1\": \"en\", \"2\": \"zh\", \"3\": \"ru\", \"4\": \"fr\", \"5\": \"de\", \"6\": \"pt\", \"7\": \"es\", \"8\": \"it\",\n \"9\": \"id\", \"10\": \"ko\", \"11\": \"ja\", \"12\": \"vi\", \"13\": \"th\", \"14\": \"ar\",\n \"15\": \"yue\", \"16\": \"hi\", \"17\": \"el\", \"18\": \"tr\"\n }\n print(\"Select target translation language (Text only mode):\")\n print(\"1. English | 2. Chinese | 3. Russian | 4. French | 5. German | 6. Portuguese | 7. Spanish | 8. Italian | 9. Indonesian | 10. Korean | 11. Japanese | 12. Vietnamese | 13. Thai | 14. Arabic | 15. Cantonese | 16. Hindi | 17. Greek | 18. Turkish\")\n\n choice = input(\"Enter option (default is the first one): \").strip()\n target_language = lang_map.get(choice, next(iter(lang_map.values())))\n\n return target_language, audio_enabled\n\n\nasync def main():\n \"\"\"Program entry point: connect, configure the session, and drive the live-translation loop.\"\"\"\n print_banner()\n\n api_key = os.environ.get(\"DASHSCOPE_API_KEY\")\n if not api_key:\n print(\"[ERROR] Please set the environment variable DASHSCOPE_API_KEY\")\n print(\" Example: export DASHSCOPE_API_KEY='your_api_key_here'\")\n return\n\n target_language, audio_enabled = get_user_config()\n print(\"\\nConfiguration complete:\")\n print(f\" - Target language: {target_language}\")\n if not audio_enabled:\n print(\" - Output mode: Text only\")\n\n client = LiveTranslateClient(api_key=api_key, target_language=target_language, audio_enabled=audio_enabled)\n\n # Callback fired as translated text arrives — stream it to stdout, character by character\n def on_translation_text(text):\n print(text, end=\"\", flush=True)\n\n try:\n print(\"Connecting to the translation service...\")\n await client.connect()\n\n # Launch the audio playback thread (only does real work when audio output is enabled)\n client.start_audio_player()\n\n print(\"\\n\" + \"-\" * 60)\n print(\"Connected! Please speak into the microphone.\")\n print(\"The program will translate your speech in real time and play the results. Press Ctrl+C to exit.\")\n print(\"-\" * 60 + \"\\n\")\n\n # Run two coroutines concurrently: server-message handling + microphone audio upload\n message_handler = asyncio.create_task(client.handle_server_messages(on_translation_text))\n tasks = [message_handler]\n # Microphone capture is the translation input source — required regardless of output mode\n microphone_streamer = asyncio.create_task(client.start_microphone_streaming())\n tasks.append(microphone_streamer)\n\n await asyncio.gather(*tasks)\n\n except KeyboardInterrupt:\n print(\"\\n\\nUser interrupted, exiting...\")\n except Exception as e:\n print(f\"\\nFatal error occurred: {e}\")\n finally:\n print(\"\\nCleaning up resources...\")\n await client.close()\n print(\"Program exited.\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n## Future Directions\n\nWe will continue exploring the capability boundaries of multimodal translation and focus on the following directions:\n\n- **Lower latency**: keep reducing end-to-end simultaneous interpretation latency toward real-time experience limits.\n- **More languages and dialects**: expand input/output coverage for low-resource languages, regional dialects, and cross-regional expressions.\n- **Longer context and stronger consistency**: maintain terminology, names, and context consistency in long meetings and multi-turn dialogues.\n- **Higher-fidelity voice cloning**: preserve speaker characteristics while restoring ambient sounds and on-site atmosphere more naturally.\n- **Richer interaction modes**: support multilingual, mixed-dialect expression, speaker separation, and joint multimodal modeling with gestures, lip movement, and expressions.\n\n## Citation\n\nFeel free to cite the following article if you find Qwen3.5-LiveTranslate helpful:\n\n```bibtex\n@misc{qwen35livetranslateblog,\n title = {Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right},\n url = {https://qwen.ai/blog?id=qwen3.5-livetranslate},\n author = {Qwen Team},\n month = {May},\n year = {2026}\n}\n```\n", "dateCreated": "2026-10-19T01:26:53.998049939+00:00", "date_created": "2026-05-19T09:40:00+00:00", "date_modified": null, "date_published": "2026-05-19T09:40:00+00:00", "description": "DashScope Demo Qwen3.5-LiveTranslate-Flash is the latest simultaneous interpretation model in the Qwen family, built on top of Qwen3.5-Omni. It delivers real-time, multimodal translation that not only hears and translates speech, but also…", "external_url": "https://qwen.ai/blog?id=qwen3.5-livetranslate", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/qwen-ai/2026/05/2026-05-19-qwen3-5-livetranslate-from-sound-to-sight-from-word-to-right.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/qwen-ai/2026/05/2026-05-19-qwen3-5-livetranslate-from-sound-to-sight-from-word-to-right.md" }, "id": "urn:aggr:item:b54e1ddcd2fb1814ed65a03dd456f106a023721e", "isBasedOn": { "@id": "https://qwen.ai/blog?id=qwen3.5-livetranslate" }, "mainEntity": { "@id": "https://qwen.ai/blog?id=qwen3.5-livetranslate", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-05-19-qwen3-5-livetranslate-from-sound-to-sight-from-word-to-right/#webpage" }, "author": [ { "@type": "Person", "name": "QwenTeam" } ], "datePublished": "2026-05-19T09:40:00+00:00", "headline": "Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right", "image": { "@type": "ImageObject", "caption": null, "height": 90, "url": "https://img.alicdn.com/imgextra/i1/O1CN01VmUTIT1HdI4ZLu25U_!!6000000000780-2-tps-1590-954.png", "width": 160 }, "timeRequired": "PT11M", "url": "https://qwen.ai/blog?id=qwen3.5-livetranslate", "wordCount": 2347 }, "name": "Archived snapshot: Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right", "preview": { "alt": null, "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://img.alicdn.com/imgextra/i1/O1CN01VmUTIT1HdI4ZLu25U_!!6000000000780-2-tps-1590-954.png", "width": 160 }, "reading_minutes": 11, "replicated_at": null, "source": "qwen-ai", "source_name": "Qwen", "summary": null, "tags": [ "release" ], "temporalCoverage": "2026-10-19T01:26:53.998049939+00:00", "title": "Qwen3.5-LiveTranslate: From Sound to Sight, From Word to Right", "url": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-05-19-qwen3-5-livetranslate-from-sound-to-sight-from-word-to-right/", "word_count": 2347 }