{ "@context": "https://schema.org", "@id": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-09-18-qwen3-8-omni-flash-omni-senses-agentic-delivery/#webpage", "@type": [ "WebPage", "ArchiveComponent" ], "articleBody": "![Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/banner-latest.en.png)\n\n## Introduction\n\nToday, we are launching **[Qwen3.8-Omni-Flash](https://www.alibabacloud.com/help/en/model-studio/qwen-omni)**, our next-generation native omnimodal model. Its core objective is to strengthen agent capabilities in real-world productivity scenarios, advancing omnimodal models from “understanding omnimodal content” to “planning tasks, calling tools, and completing creative work.” Building on general agentic capabilities in coding, text-based knowledge work, and GUI operation, Qwen3.8-Omni-Flash further extends agentic applications centered on audio and video, delivering strong results across workflows such as video editing, music video creation, film production and commentary, audio-visual summarization, and real-time conversations.\n\n![Qwen3.8-Omni-Flash and its applications in production](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/production-overview-v2.png#center)\n\nFigure 1. Qwen3.8-Omni-Flash and its applications in production.\n\n- **Qwen3.8-Omni-Flash** — now available on the [Qianwen AI Platform](https://www.qianwenai.com/):\n - Text, image, audio, and video inputs with a **1M-token context window**.\n\nQwen3.8-Omni-Flash supports a 1M-token context window while maintaining text performance comparable to a text-only model of the same size and delivering significant improvements in omnimodal capabilities. Across 29 evaluations1, its average score improves by more than 25% over Qwen3.5-Omni-Plus; the API price per hour of audio input decreases by more than 98%, and the price per hour of audio-visual input decreases by more than 93%2. For audio-visual agents, coding, and long-horizon tasks, the model improves by 36.5 points on WildClawBench-MM and 22.3 points on AgenticVBench, while scoring a strong 69.6 on UniClawBench. Its core capabilities also improve significantly in long-form audio and audio-visual understanding, audio-visual reasoning, audio-visual captioning, and multi-speaker recognition. For example, it gains 8.3 points on LongAudioSpan and 9.6 points on OmniVideoBench; its OmniCap-IF CSR and ISR improve by 8.5 and 14.1 points, respectively; and its AliMeeting DER and cpWER decrease from 88.11 / 89.61 to 3.35 / 17.18. By scaling data, context, and agentic environments, Qwen3.8-Omni-Flash achieves **audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash**. These advances also mean that audio and video are evolving from perceptual inputs into core media through which agents understand their environment, reason, and execute tasks.\n\n1\\. The scope includes audio reasoning benchmarks: AliMeeting-test, AISHELL-4, MagicData-RAMC, MLC-SLM (en), WenetSpeech (Net | Meeting), FLEURS-60 ASR, FLEURS-60 S2TT, SpotSoundBench, MMAU, MMAR, MMSU, MuchoMusic-RUL, HumMusQA, MusTBench, Audio-MultiChallenge, WildSpeech, and VoiceBench; audio-visual reasoning benchmarks: DailyOmni, WorldSense, AVUT, JointAVBench, OmniCloze, OmniCap-IF, QIVD, OmniVideoBench, and StreamingBench; and audio-visual agent benchmarks: WildClawBench-MM, UniClawBench, and OmniGAIA.\\\n2\\. Pricing methodology: hourly audio or audio-visual input prices are estimated as 30 times the input cost of two minutes of source material; audio-visual input uses 720p at 1 fps. Gemini 3.8 Flash uses media\\_resolution=high, Seed 2.0 Lite uses max\\_frame\\_tokens=384, and all other API parameters use their default values. Text input and output prices are in CNY per 1M tokens; Gemini and Muse prices are converted from USD at an exchange rate of 1 USD = 6.7191 CNY.\n\n![Qwen3.8-Omni-Flash pricing and representative benchmark comparison](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/performance-overview-v3.en.png)\n\nAudio and video are important media for bringing agents into real-world productivity scenarios, but they also introduce a new set of system-level challenges. Long-form audio and video are costly to store, transmit, and process across multiple rounds of inference; existing agent harness frameworks lack native support for these modalities; and workflows that connect omnimodal understanding with end-to-end task execution are still at an early stage. Addressing these challenges requires models, harness tools, and runtime environments to evolve together.\n\nTo address these challenges, we use Qwen3.8-Omni-Flash to explore how to connect source understanding, task planning, tool execution, and result delivery into a complete pipeline. It supports end-to-end, long-horizon workflows such as video editing, translation, film commentary, and content creation, advancing Omni from audio-visual understanding toward autonomous action and task completion.\n\nTo this end, we have further expanded Qwen-MM-Plugins with on-demand perception, tool use, and workflow execution for long-form audio and video. We have also open-sourced Qwen-Live Harness as a native runtime for continuous, real-time omnimodal interaction. Together, they address long-horizon workflows and real-time interaction while continuing to expand the capabilities of omnimodal agents alongside the model.\n\n[PLUGIN Qwen-MM-Plugins The gateway to audio-visual productivity—connecting multimodal understanding, content creation, and agent harnesses. Get started →](https://qwen.ai/blog?id=qwen3.8-omni-flash#IbE99) [HARNESS Qwen-Live Harness The gateway to real-time interaction—connecting audio-visual conversations, task delegation, memory, and context management. Get started →](https://qwen.ai/blog?id=qwen3.8-omni-flash#iTpXF)\n\n## Long-Form Audio-Visual Understanding\n\nQwen3.8-Omni-Flash brings a major upgrade to long-form audio-visual understanding—from controllable descriptions and agentic evidence gathering, to understanding meetings and advancing follow-up tasks, and finally to producing video-centered deep research reports. It does not merely process longer content, but finds relevant evidence more precisely, reasons more deeply, and acts more efficiently.\n\n### Controllable Audio-Visual Captioning\n\nThere is no single answer to how a video should be described. Content creation prioritizes narrative, footage retrieval focuses on specific segments, and asset management depends on structure. Different applications need different video descriptions. In Qwen3.8-Omni-Flash, we have upgraded video captioning from answering \"what the model saw\" to understanding \"what the user wants to know.\" Users can freely specify the subject, time range, level of detail, and output format. For the same video, the model can provide an overview, locate key segments, or **analyze character actions, camera shots, lighting, and sound in depth, producing structured results as needed**. You define what to look at, how closely to look, and how to present it.\n\n### Agentic Long-Form Audio-Visual Understanding\n\nFor videos lasting several hours, conventional approaches require the model to process the entire recording from beginning to end, even when the answer appears in only a few minutes of footage. The native Qwen3.8-Omni-Flash agent starts from the question, **independently decides what to watch and listen to, and locates key information through multiple rounds of coarse-to-fine evidence gathering**. Without processing every frame, it can focus limited compute and token budgets on the relevant segments, enabling more efficient long-form video understanding. On OmniVideoBench, Agentic Understanding **improves accuracy from 63.4 to 67.8** while reducing token consumption from 145,736 to 79,117, **a reduction of approximately 45.7%**. The table below compares the accuracy and token consumption of Static Understanding and Agentic Understanding on OmniVideoBench:\n\nNote: Agentic mode preserves context across turns.\n| | Static Understanding | Agentic Understanding |\n| -------------------- | -------------------- | --------------------- |\n| Accuracy (↑) | 63.4 | 67.8 |\n| Tokens per query (↓) | 145,736 | 79,117 |\n\n### Long Meetings: From Minutes to Action\n\nMulti-participant meetings are among the most complex audio-visual understanding scenarios: speakers take turns and overlap, while identities, references, and discussion topics continuously change. Qwen3.8-Omni-Flash **jointly recognizes speakers across audio and video** and natively supports **up to one hour** of audio-visual input. It can **perform speaker segmentation, content transcription, and identity alignment end to end**. Given a complete meeting video and a request, the model can map participant relationships, generate meeting minutes, identify action items, and analyze project risks, using visual information to resolve references and entity ambiguity in the audio. Combined with agents and tool use, it can also send emails, organize tasks, and even begin coding in response to meeting requirements—moving from understanding a meeting to acting on it.\n\n### Conducting Deep Research with Audio and Video\n\nWhen users watch a video with a specific question in mind, the answer often extends beyond the video itself. Qwen3.8-Omni-Flash combines the user’s needs with the video content to identify questions worth deeper investigation, organize the key material, and search multimodal sources across the web—including images, videos, and documents. It then produces a video-centered, richly illustrated research report that helps users understand the content and solve practical problems. For example, when a user encounters color fringing around a Photoshop hair cutout, the model can break down the tutorial steps, study the principles behind Multiply and Screen blend modes, compare alternative edge-repair techniques, and explain which approach best fits the user’s situation.\n\n[Qwen3.8-Omni-Flash Conducting Deep Research with Audio and Video Demo](https://docs.qwenlm.ai/resources/qwen3.8-omni-flash/omnidr-demo-0813-0600.51939eff.html)\n\n## Audio-Visual Production and Editing\n\nQwen3.8-Omni-Flash is taking audio-visual agents into a new stage: **from understanding sounds and images to independently planning, calling tools, and delivering finished videos**, bringing omnimodal intelligence into professional audio-visual content production workflows.\n\n### Music2MV\n\nFor music video (MV) creation, Qwen3.8-Omni-Flash can understand the structure, rhythm, mood, vocals, and instrumental changes of a user-provided song in fine detail, informing the design of characters, scenes, and shots. It can also output line-level lyrics with timestamps to align singing, subtitles, and visuals. Combined with creative tools such as Qwen-MM-Plugins, the model supports the complete workflow from music understanding and creative planning to final quality review, demonstrating strong audio-visual understanding, reasoning, and creation capabilities.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nFinal Result 2 Next\n\nFinal Result 3 Next\n\n### Short Drama Translation\n\nTraditional video translation often requires repeatedly switching between transcription, translation, dubbing, and editing platforms. This complicates API calls and workflow coordination and makes it difficult to maintain consistency across character voices, dialogue duration, and visual pacing. With an agent built on Qwen3.8-Omni-Flash, users can describe their needs in a single sentence to perform speaker-aware dialogue recognition, conversational translation, character voice cloning and dubbing, audio remixing, and final quality review. These otherwise fragmented localization steps become a complete workflow, enabling automated delivery of short dramas for international audiences.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nSource\n\nTranslated Result\n\nFinal Result 2 Next\n\nSource\n\nTranslated Result\n\nFinal Result 3 Next\n\nSource\n\nTranslated Result\n\n### Long-Form Film Commentary\n\nProducing commentary videos for full-length films of two or three hours often requires repeatedly watching the film and reconstructing its plot, followed by shot selection, scriptwriting, voiceover, music, and editing. This is a complex and time-consuming process. With an agent built on Qwen3.8-Omni-Flash, users need only provide a film and describe their creative requirements in one sentence to perform long-form video understanding, key-plot extraction, commentary planning, voiceover and music production, editing, rendering, and final quality review. The agent can also intelligently interleave original dialogue with commentary, automatically adjusting speech rate and volume so that narration, original audio, background music, and visuals flow naturally together, creating a more authentic, immersive, and cinematic commentary video.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nFinal Result 2 Next\n\n## From Using Models to Optimizing Models\n\nReal-world multimodal applications are often complex and cost-sensitive, requiring models to balance quality, latency, compute, and deployment costs.\n\nCustomizing smaller models for specific scenarios is therefore an important path to deploying applications at scale. Yet traditional workflows involve data construction, problem diagnosis, multiple rounds of training, and evaluation, making them time-consuming and heavily dependent on human expertise. This time, we extend Qwen3.8-Omni-Flash into model development itself, exploring a new approach in which large models drive research and development while smaller models serve business needs.\n\nWe gave Qwen3.8-Omni-Flash a task: **improve Qwen2.5-Omni-3B's Sichuan dialect speech recognition within 12 hours and deliver a usable model**. It independently selected the WenetSpeech-Chuan evaluation set, fixed the evaluation criteria, and established a baseline. It then listened directly to audio samples, diagnosed problems using the recognition results, and constructed targeted training data. Across four consecutive rounds of experiments, the agent created 3,413 training examples, adjusted its approach based on evaluation feedback, retained effective improvements, and rolled back unsuccessful attempts. **Qwen2.5-Omni-3B's character error rate on the same evaluation set ultimately fell from 25.79% to 15.30%, a relative reduction of approximately 40.7%**.\n\nThis experiment demonstrates another possibility for model evolution: general-purpose multimodal models understand data, plan experiments, and drive iteration, while smaller models acquire specialized capabilities for specific applications. Agents can go beyond using models to help solve practical business problems.\n\n## Audio-Visual Information Compression\n\nAudio and video carry rich information, but their linear, unstructured form makes retrieval and reuse difficult. Qwen3.8-Omni-Flash understands content across sound, visuals, and timelines, using an agentic workflow to extract information, reorganize its structure, and verify results. **It transforms the core knowledge and practical experience in long videos into denser information assets that are easier to consume and reuse**.\n\n### Video2Note\n\nTo turn video knowledge into structured resources, we have open-sourced Video2Note in Qwen-MM-Plugins. Drawing on Qwen3.8-Omni-Flash's joint understanding of speech, visuals, and procedures, it automatically organizes knowledge, breaks down key steps, selects representative frames, and generates PDF notes with corresponding text and images. Automated review and iterative correction further condense hours of video into clear, readable documents that are easy to revisit.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 2 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 3 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 4 Next\n\nExample Video\n\nFinal PDF\n\n### Omni Skill Creator\n\nVideos record not only \"how to do something,\" but also the practical expertise accumulated by specialists. With this in mind, we introduce Omni Skill Creator as a new open-source capability in Qwen-MM-Plugins. It can extract standard operating procedures (SOPs) from demonstrations to perform reusable automated work, or learn tool usage, decision criteria, and key insights from expert instruction. A single demonstration becomes an agent skill that has been verified and evaluated, enabling reusable, shareable skills built from omnimodal content.\n\n## Real-Time Audio-Visual Interaction\n\nQwen3.8-Omni-Flash is designed for deep understanding and creation with complete audio-visual content. For continuous, low-latency interaction, we further introduce **Qwen3.8-Omni-Flash-Realtime**. It perceives and responds while receiving live audio-visual streams, and uses real-time context to call tools and execute tasks, taking omnimodal capabilities from “understanding a piece of content” to “participating in an interaction.”\n\n### Real-Time Speaking Practice\n\nSpoken language has no standard input. Accents, vowel and consonant substitutions, and tonal deviations can cause word-for-word transcription to diverge from the intended meaning. Qwen3.8-Omni-Flash-Realtime jointly models pronunciation and semantics, understands nonstandard expressions affected by accents, aligns them with the correct words, and generates standard-pronunciation demonstrations in real time. Across multiple practice turns, the model updates its judgment with new audio, correcting errors that affect understanding while preserving natural rhythm, tone, and emotion.\n\n### Omnimodal Spatial Audio Perception\n\nIn real-world spaces, sound provides another coordinate axis beyond vision. Qwen3.8-Omni-Flash-Realtime combines spatial sound with visual information to continuously determine the direction and distance of sound sources while perceiving obstacles, navigable areas, and changes in the scene, making it the first omnimodal model capable of locating targets by sound.\n\nFor instructions such as “come over here” or “go see what is making that sound,” the model can isolate voices and target sounds from environmental noise, ground their meaning in the surrounding space, and call tools to perform localization, search, path planning, and navigation—from hearing a target to reaching it.\n\n### External Knowledge for Audio-Visual Interaction\n\nReal-time interaction requires not only low latency, but also the ability to load knowledge and behavior dynamically for each application. Qwen3.8-Omni-Flash-Realtime supports injecting identity settings, expression styles, business knowledge, and interaction rules through Skills, while tool use extends these capabilities into task execution.\n\nIn scenarios such as customer service, the model can load brand language and service procedures in real time, understand the user's speech, visuals, and context, generate responses that follow business requirements, and execute actions. The same real-time model can therefore take on different knowledge, roles, and ways of acting.\n\n## Benchmark Results\n\n### Omni\n\n| | Qwen3.8-Omni-Flash | Qwen3.5-Omni-Plus | Gemini 3.8 Flash | Seed 2.0 Lite | Muse Spark 1.2 |\n| --------------------------------------------------- | ------------------------------------------------- | ----------------------------------------- | --------------------------------------------- | -------------------- | -------------------- |\n| Agentic Omni Intelligence | | | | | |\n| WildClawBench-MM Multimodal tool use | **71.0** | 34.5 | 58.9 | 41.9 | -- |\n| UniClawBench Multimodal tool use | **69.6** | 67.1 | 69.0 | 61.2 | -- |\n| AgenticVBench Multimodal tool use | 36.8 | 14.5 | **45.0** | 10.0 | -- |\n| OmniGAIA Web Search | 74.0 | 57.2 | **78.6** | 64.4 | -- |\n| General Audio-Visual Capabilities | | | | | |\n| DailyOmni Audio-Visual Understanding | **85.1** | **85.1** | 84.0 | 81.4 | 79.6 |\n| WorldSense Audio-Visual Understanding | 68.5 | 63.9 | **69.6** | 67.3 | 65.0 |\n| AVUT Audio-Visual Understanding | 86.6 | 85.9 | **88.0** | 81.5 | 82.4 |\n| JoinAVBench Audio-Visual Understanding | **75.9** | 74.1 | 70.4 | 70.6 | 71.8 |\n| OmniVideoBench Audio-Visual Reasoning | 63.4 | 53.8 | **65.2** | 58.5 | 62.2 |\n| Video-MME-v2 Audio-Visual Reasoning | 65.0 | 47.9 | **71.0** | 64.9 | -- |\n| LVOmniBench Long Video Reasoning | 63.3 | 53.2 | **70.7** | -- | -- |\n| OmniCloze Audio-Visual Caption | 63.2 | 64.2 | 60.9 | 56.3 | **65.3** |\n| OmniCap-IF Audio-Visual Caption | CSR 80.6 ISR 28.2 | CSR 72.1 ISR 14.1 | CSR **81.9** ISR **28.3** | CSR 74.6 ISR 18.1 | CSR 77.9 ISR 26.8 |\n| QIVD Audio-Visual Interaction | **69.6** | 65.6 | 69.1 | 62.0 | 62.0 |\n| StreamingBench Audio-Visual Interaction | **80.8** | 57.1 | 79.9 | 77.2 | 77.8 |\n| General Audio Capabilities | | | | | |\n| AliMeeting Test Multi-Speaker ASR (DER | cpWER, ↓) | **3.4** | **17.2** | 88.1 | 89.6 | 72.6 | 53.1 | 75.1 | 76.1 | 93.7 | 92.7 |\n| AISHELL-4 Multi-Speaker ASR (DER | cpWER, ↓) | **2.8** | **11.2** | 100.0 | 100.0 | 66.4 | 56.9 | 64.8 | 64.2 | 91.3 | 86.0 |\n| MagicData-RAMC Multi-Speaker ASR (DER | cpWER, ↓) | **5.7** | **14.1** | 98.4 | 97.1 | 67.9 | 33.8 | 43.4 | 35.1 | 82.1 | 75.3 |\n| MLC-SLM (en) Multi-Speaker ASR (DER | cpWER, ↓) | **4.0** | **14.2** | 68.6 | 63.9 | 60.8 | 26.6 | 40.4 | 45.5 | 74.3 | 52.9 |\n| WenetSpeech (Net) ASR (WER, ↓) | 4.8 | **3.7** | 14.2 | 4.3 | 68.2 |\n| WenetSpeech (Meeting) ASR (WER, ↓) | **4.6** | 4.8 | 16.7 | 4.7 | 42.6 |\n| FLEURS-ASR Multilingual ASR (WER, ↓) | 9.3 | **7.2** | 7.9 | 32.1 | 23.6 |\n| FLEURS-S2TT Multilingual S2TT (BLEU) | 31.8 | 32.2 | **33.0** | 24.8 | 28.8 |\n| SpotSoundBench Audio Grounding | **67.2** | 64.2 | 39.7 | 59.6 | 16.9 |\n| MMAU Audio Understanding | 81.8 | **81.9** | 76.9 | 77.2 | 63.5 |\n| MMAR Audio Understanding | **79.8** | **79.8** | 78.5 | 77.7 | 67.3 |\n| MMSU Audio Understanding | 82.1 | 83.0 | **83.3** | 80.2 | 59.9 |\n| LongAudioSpan Long Audio Reasoning | Accuracy **82.7** Rubric **71.8** Chain 48.2 | Accuracy 74.4 Rubric 49.8 Chain 45.1 | Accuracy 79.3 Rubric 65.5 Chain **64.6** | -- | -- |\n| MuchoMusic-RUL Music Understanding | **72.6** | 71.6 | 53.7 | 61.7 | 40.1 |\n| HumMusQA Music Understanding | **75.8** | 75.5 | 71.2 | 66.0 | 63.3 |\n| MusTBench Music Understanding | **50.6** | 49.1 | 40.3 | 44.0 | 29.4 |\n| Audio MultiChallenge Audio Interaction | 71.5 | 57.6 | **71.9** | 63.4 | 57.9 |\n| WildSpeech Audio Interaction | 74.3 | 75.7 | **76.4** | 74.5 | 73.4 |\n| VoiceBench Audio Interaction | 91.6 | **92.9** | 92.3 | 84.1 | 79.8 |\n\n1\\. Evaluation harnesses for Agentic Omni Intelligence: WildClawBench-MM and AgenticVBench use Claude Code, UniClawBench uses OpenClaw, and OmniGAIA uses no harness. For WildClawBench-MM, we evaluate only the multimodal tasks in WildClawBench that involve images, video, or audio.\\\n2\\. FLEURS: ASR and S2TT evaluation results both cover the following 60 languages: Chinese (Mandarin), English, Cantonese, Arabic, German, French, Spanish, Portuguese, Indonesian, Italian, Korean, Russian, Thai, Vietnamese, Japanese, Turkish, Hindi, Malay, Dutch, Urdu, Norwegian, Swedish, Danish, Hebrew, Finnish, Polish, Icelandic, Czech, Filipino, Persian, Greek, Afrikaans, Asturian, Belarusian, Bulgarian, Bengali, Bosnian, Catalan, Cebuano, Estonian, Galician, Gujarati, Croatian, Hungarian, Javanese, Kazakh, Kannada, Kyrgyz, Latvian, Macedonian, Malayalam, Marathi, Punjabi, Romanian, Slovak, Slovenian, Swahili, Tajik, Azerbaijani, and Ukrainian.\\\n3\\. Empty cells (--): scores are not yet available or are not applicable.\n\n### Agentic Omni Understanding\n\nKey information in long audio and video recordings is often scattered across different segments, while complex questions require multiple steps of reasoning across sound and images. Agentic Omni Understanding enables the model to start from the question, plan its approach, call tools, and progressively locate and verify evidence. By focusing computation on relevant content, it improves the accuracy and efficiency of long-form audio-visual understanding. To evaluate this capability, we compare Qwen3.8-Omni-Flash and Gemini 3.8 Flash on OmniVideoBench, Video-MME-v2, and LVOmniBench under two settings: **Static**, where the model directly interprets the input, and an agent mode using **[Qwen Code](https://github.com/qwenLM/qwen-code)**. This comparison shows how introducing agent workflows affects each model’s performance.\n\n| | Qwen3.8-Omni-Flash (Static) | Qwen3.8-Omni-Flash (Qwen Code) | Gemini 3.8 Flash (Static) | Gemini 3.8 Flash (Qwen Code) |\n| -------------------------------------- | --------------------------- | ------------------------------ | ------------------------- | ---------------------------- |\n| OmniVideoBench Audio-Visual Reasoning | 63.4 | 67.8 | 65.2 | **70.1** |\n| Video-MME-v2 Audio-Visual Reasoning | 65.0 | 71.3 | 71.0 | **72.7** |\n| LVOmniBench Long Video Reasoning | 63.3 | **73.6** | 70.7 | 70.7 |\n\n### Text\n\n| | Qwen3.8-Omni-Flash | Qwen3.8- Flash | Qwen3.8- 27B | Qwen3.7- Plus | DeepSeek-V4-Flash-0731 | Claude-Opus-4.6 (Max) |\n| --------------------------------------------------------- | ------------------ | -------------- | ------------ | ------------- | ---------------------- | --------------------- |\n| Coding and Agent | | | | | | |\n| DeepSWE 1.1 Long-horizon software engineering | 57.8 | **58.7** | 42.2 | 16.5 | 54.4 | -- |\n| SWE-bench Pro Long-horizon software engineering | **63.3** | 62.5 | 61.7 | 55.8 | 56.0 | 53.4 |\n| SWE-bench Multilingual Multilingual software engineering | 80.5 | **81.0** | 73.8 | 75.8 | -- | 77.5 |\n| NL2Repo-Bench Repo-level code generation | 48.9 | 48.1 | 42.3 | 41.1 | **54.2** | 47.6 |\n| CoWorkBench Long-horizon office work | **75.3** | 73.9 | 70.7 | 65.1 | 45.1 | 68.2 |\n| General Text Capabilities | | | | | | |\n| IFBench Instruction following | **81.5** | 81.3 | 79.5 | 79.1 | 79.2 | 62.5 |\n| GPQA Diamond Scientific reasoning | 91.0 | **91.7** | 89.2 | 90.3 | 90.8 | 91.3 |\n| HLE Multidisciplinary reasoning | 36.5 | 35.9 | 30.8 | 34.7 | 33.8 | **40.0** |\n| LiveCodeBench v6 Competitive coding | **92.6** | 91.9 | 90.3 | 89.6 | 90.6 | 88.8 |\n\n1\\. DeepSWE 1.1: evaluated with the Claude Code and mini-SWE-agent harnesses, temp=1.0, top\\_p=0.95, 256K context window. We report the highest score across the two harnesses; notably, Qwen3.8-Flash-Next performs best on mini-SWE-agent.\\\n2\\. SWE-bench Pro: except for Claude-Opus-4.6 (Max), for which we report the officially published score, all models are evaluated with the Claude Code harness, temp=1.0, top\\_p=0.95, 256K context window. Problematic tasks were corrected and all baseline models were re-evaluated on the refined benchmark.\\\n3\\. SWE-bench Multilingual: evaluated with the mini-SWE-agent harness, temp=1.0, top\\_p=0.95, 256K context window.\\\n4\\. NL2Repo-Bench: evaluated with the Claude Code harness. To prevent reward hacking, we disable Bash commands that attempt to access the specific repository, such as pip download, pip install and git clone.\\\n5\\. CoWorkBench: an in-house cowork benchmark for evaluating long-horizon office and productivity agent tasks across computer science, finance, law, medical and other productivity domains.\\\n6\\. HLE: judged by GPT-4o.\\\n7\\. Empty cells (--): scores are not yet available or are not applicable.\n\n### Vision\n\n| | Qwen3.8-Omni-Flash | Qwen3.8- Flash | Qwen3.8- 27B | Qwen3.7- Plus | Claude-Opus-4.6 (Max) |\n| --------------------------------------- | --------------------------------------- | ------------------------------- | ------------------------------- | ----------------------------------- | ----------------------------- |\n| Agentic Vision Intelligence | | | | | |\n| ClawEval-MM Multimodal tool use | Pass@3 60.4 Average **61.9** | Pass@3 **64.4** Average 60.4 | Pass@3 57.4 Average 56.9 | Pass@3 57.4 Average 60.1 | Pass@3 52.5 Average 54.7 |\n| AndroidWorld Mobile use | **87.1** | 84.5 | 81.9 | 81.0 | 62.0 |\n| Vision2Web Visual web development | 62.9 | **64.0** | 62.9 | 42.1 | -- |\n| General Vision Capabilities | | | | | |\n| ERQA Embodied intelligence | 71.0 | **72.3** | 65.5 | 69.8 | 40.8 |\n| LVBench Long video understanding | **76.9** | 76.6 | 72.4 | 76.2 | 63.0 |\n| RealWorldQA Real-world perception | 87.7 | **88.5** | 85.9 | 86.9 | 73.9 |\n| MathVision Visual math problem solving | Without CI **91.8** With CI **96.2** | Without CI 90.6 With CI 95.7 | Without CI 90.0 With CI 94.6 | Without CI 90.3 With CI 88.4 | Without CI 65.5 With CI -- |\n| CharXiv (RQ) Scientific chart analysis | Without CI 83.5 With CI **91.4** | Without CI 84.6 With CI 90.6 | Without CI 83.7 With CI 90.2 | Without CI **85.8** With CI 85.9 | Without CI 66.0 With CI -- |\n\n1\\. ClawEval-MM: scores are reported as \"pass@3 / average score\". Pass@3 measures the percentage passed in at least one of three trials, and the average score is the mean score across the three trials.\\\n2\\. RecreationBench: an in-house long-horizon application-recreation benchmark for evaluating hybrid-agent abilities spanning five platforms — desktop (Ubuntu, macOS, Windows), mobile (Android) and web.\\\n3\\. OSWorld 2.0: scores are reported as \"binary / partial\". The binary score is the percentage of tasks that receive the full task reward, while the partial score aggregates the partial rewards obtained across all tasks.\\\n4\\. Vision2Web: scores are reported as the average over the frontend, webpage and website categories, using the Claude Code harness and judged by gpt-5.4-2026-03-05.\\\n5\\. MathVision, CharXiv (RQ): scores are reported as \"without CI / with CI\". A small number of incorrect ground-truth annotations in MathVision were corrected after manual verification. Our model's score is evaluated using a fixed prompt, e.g. \"Please reason step by step, and put your final answer within \\\\boxed{}.\" For other models, we report the higher score between runs with and without the \\\\boxed{} formatting.\\\n6\\. Empty cells (--) indicate scores not yet available or not applicable.\n\n### Throughput & Latency\n\nThe following results show the observed throughput and latency of the Qwen3.8-Omni-Flash-Realtime API under different input conditions, reflecting the performance users experience in production environments.\n\n| Input Scenario | Text Output TPS (Tokens/s) | Time to First Token (ms) | Time to First Audio Packet (ms) | Audio Generation RTF |\n| ------------------------ | -------------------------- | ------------------------ | ------------------------------- | -------------------- |\n| Realtime API Performance | | | | |\n| Audio 6s | 84.87 | 591.26 | 978.36 | 0.1538 |\n| Audio 12s | 83.18 | 604.80 | 982.74 | 0.1537 |\n| Audio 20s | 81.06 | 617.98 | 1026.39 | 0.1538 |\n| Audio-Visual 6s | 84.89 | 837.96 | 1214.73 | 0.1524 |\n| Audio-Visual 12s | 84.33 | 911.85 | 1268.07 | 0.1527 |\n| Audio-Visual 20s | 83.00 | 981.01 | 1350.49 | 0.1528 |\n\n### Supported Languages\n\n| Capability | Languages | Chinese Dialects |\n| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Speech Recognition** | **74 languages:** Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Norwegian Nynorsk, Oriya, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajiki, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, and Vietnamese | **39 dialects:** Northeastern Mandarin, Guizhou dialect, Guangdong Cantonese, Henan dialect, Hong Kong Cantonese, Shanghainese, Shaanxi dialect, Tianjin dialect, Taiwanese Mandarin, Yunnan dialect, Anhui dialect, Fujian dialect, Gansu dialect, Guangdong Mandarin, Hubei dialect, Hunan dialect, Jiangxi dialect, Shandong dialect, Shanxi dialect, Sichuanese, Guangxi dialect, Hainan dialect, Chongqing dialect, Changsha dialect, Hangzhou dialect, Hefei dialect, Yinchuan dialect, Zhengzhou dialect, Shenyang dialect, Wenzhou dialect, Wuhan dialect, Kunming dialect, Taiyuan dialect, Nanchang dialect, Jinan dialect, Lanzhou dialect, Nanjing dialect, Hakka, and Southern Min |\n| **Speech Generation** | **29 languages:** Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Tagalog, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, and Persian | **7 dialects:** Sichuanese, Beijing dialect, Tianjin dialect, Nanjing dialect, Shaanxi dialect, Cantonese, and Southern Min |\n\n## Getting Started with Qwen3.8-Omni-Flash\n\n### API Usage\n\nQwen3.8-Omni-Flash officially supports `reasoning_effort` to adjust reasoning depth and control costs:\n\n- `xhigh` (default): for complex tasks that require in-depth analysis.\n- `medium`: balances accuracy and speed.\n- `low`: efficient reasoning optimized for speed and cost.\n\nIn addition, `preserve_thinking` is enabled by default across all scenarios for the best out-of-the-box experience.\n\nQwen3.8-Omni-Flash supports industry-standard protocols, including OpenAI-compatible Chat Completions and Responses APIs. Examples follow:\n\n```python\n\"\"\"\nEnvironment variables:\n DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n DASHSCOPE_BASE_URL: (optional) Base URL for compatible-mode API.\n - Beijing: https://dashscope.aliyuncs.com/compatible-mode/v1\n - Singapore: https://dashscope-intl.aliyuncs.com/compatible-mode/v1\n\"\"\"\nfrom openai import OpenAI\nimport os\n\napi_key = os.environ.get(\"DASHSCOPE_API_KEY\")\nif not api_key:\n raise ValueError(\n \"DASHSCOPE_API_KEY is required. \"\n \"Set it via: export DASHSCOPE_API_KEY='your-api-key'\"\n )\n\nclient = OpenAI(\n api_key=api_key,\n base_url=os.environ.get(\n \"DASHSCOPE_BASE_URL\",\n \"https://dashscope-intl.aliyuncs.com/compatible-mode/v1\",\n ),\n)\n\nmessages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg\"\n },\n },\n {\n \"type\": \"input_audio\",\n \"input_audio\": {\n \"data\": \"https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav\",\n \"format\": \"wav\"\n },\n },\n {\"type\": \"text\", \"text\": \"Please describe the image and tell me what is being said in the audio.\"},\n ],\n },\n]\n\ncompletion = client.chat.completions.create(\n model=\"qwen3.8-omni-flash\",\n messages=messages,\n extra_body={\n \"enable_thinking\": True,\n # \"preserve_thinking\": True,\n },\n reasoning_effort=\"xhigh\", # supported levels are xhigh, medium, and low\n stream=True,\n)\n\nreasoning_content = \"\"\nanswer_content = \"\"\nis_answering = False\nprint(\"\\n\" + \"=\" * 20 + \"Reasoning\" + \"=\" * 20 + \"\\n\")\n\nfor chunk in completion:\n if not chunk.choices:\n print(\"\\nUsage:\")\n print(chunk.usage)\n continue\n\n delta = chunk.choices[0].delta\n\n if hasattr(delta, \"reasoning_content\") and delta.reasoning_content is not None:\n if not is_answering:\n print(delta.reasoning_content, end=\"\", flush=True)\n reasoning_content += delta.reasoning_content\n\n if hasattr(delta, \"content\") and delta.content:\n if not is_answering:\n print(\"\\n\" + \"=\" * 20 + \"Answer\" + \"=\" * 20 + \"\\n\")\n is_answering = True\n print(delta.content, end=\"\", flush=True)\n answer_content += delta.content\n```\n\n### Best Practices for Controllable Audio-Visual Captioning\n\n**Content organization and level of detail:** Organize video descriptions, OCR extraction, audio descriptions, and speech transcripts into sections covering visuals, speech, music, sound effects, and ambient sounds. Preserve the original text, speaker identities, and corresponding time ranges to make retrieval and verification easier. Specify a target length in the prompt to control detail, for example, `Describe the video in approximately 2000–3000 words.`, and adjust it to the video’s duration and information density.\n\nPrompt Example\n\n````text\nProvide a detailed description of the video.\n\nMake sure your description covers every one of the following dimensions:\n\nVisual\n- Subjects and characters: appearance, clothing, gender/age cues, identity, distinctive features\n- Actions and events in chronological order, and how the scene evolves over time\n- Setting and background: location, environment, time of day\n- Spatial layout and relations between subjects/objects; counts and quantities\n- On-screen text: captions, titles, subtitles, logos, UI — exact content and appearance\n- Visual style: colors, lighting, camera shots, angles, and camera movement\n\nAudio\n- Speech: the exact spoken content, transcribed verbatim\n- Speakers: who is speaking (mapped to the on-screen person or voice-over), with accent, tone, gender/age cues\n- Speaking state: prosody, emotion, volume, and speaking style\n- Music: presence, genre/mood, and lyrics if any\n- Sound effects and ambient/background sounds\n- Non-speech vocalizations: laughter, crying, applause, etc.\n\nAudio-visual correspondence\n- Which speech or sound aligns with which on-screen person or visual event\n- The timing of each event, expressed with timestamps\n\nIt should explicitly include three sections:\n\n1. A structured chronological storyline of **every noticeable audio and visual details**\n2. A structured list of all visible text. For each text element, include start timestamp, end timestamp, the exact text content, the appearance characteristics. If no text appears, explicitly state so.\n3. A structured speech-to-text transcription, include speaker (corresponding to the character or voice-over in Section 1, including their accent and tone), exact spoken content, start timestamp, end timestamp, and speaking state (prosody, emotion, and style). If no speech appears, explicitly state so.\n\nAside from these three required sections, you are free to organize any additional content in any way you find helpful. This additional content can include global information about the entire video or localized information about specific moments. You may choose the topic of this extra content freely.\n\nRules:\n\n- Add as much descriptive detail as possible.\n- Do not use Markdown bold formatting.\n- Carefully look at frames and listen to the audio, making sure no detail is overlooked.\n\nOutput Format:\n```\n## Storyline\n\n - \n\n\n - \n\n\n - \n\n\n...\n\n## Visible Text\n\n - \n“”: \n“”: \n\n - \n“”: \n“”: \n“”: \n\n - \n“”: \n\n...\n\n## Speakers and Transcript\n\nSpeaker profiles:\n - \n - \n - \n...\n\n - \nSpeaker: \nState: \nContent: “”\n\n - \nSpeaker: \nState: \nContent: “”\n\n - \nSpeaker: \nState: \nContent: “”\n\n...\n\n## \n\n\n\n## \n\n\n\n...\n```\n````\n\n**Structured output and schema constraints:** Include task instructions and a complete JSON Schema in the prompt, specifying field meanings, types, required fields, and constraints to support programmatic parsing and downstream use.\n\nPrompt Example\n\n```text\nDescribe the audio and visual content in detail in English, organized into scenes and events, following the JSON Schema below.\nAll timestamps must be relative to the beginning of the video. End times must not precede start times or exceed the video duration. Each event must fall within the time range of its parent scene.\nInclude only information directly supported by the audio or video. Do not guess or invent details. Do not infer causality merely because a sound and an action occur at the same time.\nReturn only valid JSON, without Markdown fences or commentary.\n\nJSON Schema:\n{\n \"$defs\": {\n \"Event\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"time_range\": {\n \"$ref\": \"#/$defs/TimeRange\",\n \"description\": \"Time range of the event\"\n },\n \"participants\": {\n \"description\": \"People, animals, or objects involved, named by observable features; use consistent names for the same participant\",\n \"items\": {\n \"type\": \"string\"\n },\n \"title\": \"Participants\",\n \"type\": \"array\"\n },\n \"action\": {\n \"description\": \"Specific actions, interactions, and observable outcomes\",\n \"title\": \"Action\",\n \"type\": \"string\"\n },\n \"sounds\": {\n \"description\": \"Sounds heard during the event; use an empty list if none are discernible\",\n \"items\": {\n \"type\": \"string\"\n },\n \"title\": \"Sounds\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"time_range\",\n \"participants\",\n \"action\",\n \"sounds\"\n ],\n \"title\": \"Event\",\n \"type\": \"object\"\n },\n \"Scene\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"time_range\": {\n \"$ref\": \"#/$defs/TimeRange\",\n \"description\": \"Time range of the scene\"\n },\n \"setting\": {\n \"description\": \"Environment, spatial layout, and main visual features\",\n \"title\": \"Setting\",\n \"type\": \"string\"\n },\n \"events\": {\n \"description\": \"Events in chronological order; use an empty list if there are none\",\n \"items\": {\n \"$ref\": \"#/$defs/Event\"\n },\n \"title\": \"Events\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"time_range\",\n \"setting\",\n \"events\"\n ],\n \"title\": \"Scene\",\n \"type\": \"object\"\n },\n \"TimeRange\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"start_seconds\": {\n \"description\": \"Start time in seconds relative to the beginning of the video\",\n \"minimum\": 0,\n \"title\": \"Start Seconds\",\n \"type\": \"number\"\n },\n \"end_seconds\": {\n \"description\": \"End time in seconds; must not precede the start time\",\n \"minimum\": 0,\n \"title\": \"End Seconds\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"start_seconds\",\n \"end_seconds\"\n ],\n \"title\": \"TimeRange\",\n \"type\": \"object\"\n }\n },\n \"additionalProperties\": false,\n \"properties\": {\n \"summary\": {\n \"description\": \"An overview of the main content of the video\",\n \"title\": \"Summary\",\n \"type\": \"string\"\n },\n \"scenes\": {\n \"description\": \"Scenes in chronological order; group continuous footage with a consistent setting into one scene\",\n \"items\": {\n \"$ref\": \"#/$defs/Scene\"\n },\n \"title\": \"Scenes\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"summary\",\n \"scenes\"\n ],\n \"title\": \"CaptionResult\",\n \"type\": \"object\"\n}\n```\n\n### Installing Qwen-MM-Plugins in an Agent Harness\n\n[Qwen-MM-Plugins](https://github.com/QwenLM/Qwen-MM-Plugins) is a multimodal plugin suite for agent harnesses. It gives agents the ability to understand images, audio, video, and documents, as well as maintain memory for long-form video and create content. It supports agent harnesses including Codex, Claude Code, Qwen Code, Gemini CLI, Qoder, CodeBuddy, and OpenClaw. We also welcome contributions from community developers.\n\nYou can enter the following request directly in your usual office agent:\n\n```text\nHelp me install the core, api, and omni-related plugins from https://github.com/QwenLM/Qwen-MM-Plugins.\n```\n\n![Installing Qwen-MM-Plugins in an office agent using natural language](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/mm-plugins-install-20260917.en.png)\n\nAlternatively, install from the command line:\n\n```bash\n# Run the official guided installer:\ncurl -fsSL https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh | bash\n```\n\nIn the menu, select:\n\n- Install\n- The agent harness you use\n- The Omni plugins you need\n\n| Plugin | Capability |\n| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |\n| `core` | Read images, video frames, PDFs, Office documents, code, data, and 3D files |\n| `api` | Image understanding, OCR, object localization, audio-visual transcription, speaker diarization, event analysis, and image segmentation |\n| `omni-chatcut` | Create MVs, long-form film commentary, and video speech translations |\n| `omni-video2note` | Turn video tutorials into PDF notes with key screenshots |\n| `omni-skill-creator` | Turn instructional videos, screen recordings, or operation demonstrations into reusable Agent Skill.md files |\n| `omni-memory` | Build memory for people, dialogue, sounds, and events in long-form videos |\n\nAfter installation, restart the agent harness or create a new task.\n\n```text\n@song.mp3 Generate a complete MV based on the song's rhythm and content.\n@short_drama.mp4 Translate the video into English while preserving the original speakers' voice characteristics where possible.\n@movie.mp4 Create a film commentary video with Chinese narration and subtitles.\n\n@weekly_report_sop.mp4 Turn this screen recording of writing a weekly report into a Skill.md file.\n@tutorial.mp4 Turn the tutorial into PDF notes with key screenshots, timestamps, and step-by-step instructions.\n@documentary.mp4 Build audio-visual memory that records people, dialogue, sounds, and important events.\n```\n\n## Getting Started with Qwen3.8-Omni-Flash-Realtime\n\n### API Usage\n\nQwen3.8-Omni-Flash-Realtime supports connections over WebSocket and WebRTC. Running the basic examples below opens the camera and microphone for a real-time audio-visual conversation. We recommend using headphones.\n\n```python\n# Run pip install websocket-client pyaudio dashscope opencv-python -U to install dependencies\n\"\"\"\nEnvironment variables:\n DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n DASHSCOPE_BASE_URL: (optional) Base URL for realtime API.\n - Beijing: wss://dashscope.aliyuncs.com/api-ws/v1/realtime\n - Singapore: wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime\n\"\"\"\nimport os\nimport base64\nimport time\nimport pyaudio\nimport cv2\nfrom dashscope.audio.qwen_omni import MultiModality, AudioFormat,OmniRealtimeCallback,OmniRealtimeConversation\nimport dashscope\n\nurl = f'wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime'\ndashscope.api_key = os.getenv('DASHSCOPE_API_KEY')\n# Determine the voice\nvoice = 'Tina'\n# Determine the model\nmodel = 'qwen3.8-omni-flash-realtime'\n# Determine the model role\ninstructions = \"You are Qwen-Omni, a helpful assistant.\"\nvideo_fps = 1\nvideo_size = (1280, 720)\nclass SimpleCallback(OmniRealtimeCallback):\n def __init__(self, pya):\n self.pya = pya\n self.out = None\n def on_open(self):\n # Initialize audio output stream\n self.out = self.pya.open(\n format=pyaudio.paInt16,\n channels=1,\n rate=24000,\n output=True\n )\n def on_event(self, response):\n if response['type'] == 'response.audio.delta':\n # Play audio\n self.out.write(base64.b64decode(response['delta']))\n elif response['type'] == 'conversation.item.input_audio_transcription.delta':\n # Streaming preview: text is the confirmed prefix, stash is the confirmed suffix\n preview = response.get('text', '') + response.get('stash', '')\n print(f\"\\r[User] {preview}\", end='', flush=True)\n elif response['type'] == 'conversation.item.input_audio_transcription.completed':\n # Transcription completed, print the final text and a new line\n print(f\"\\r[User] {response['transcript']}\")\n elif response['type'] == 'response.audio_transcript.done':\n # Print the assistant's response text\n print(f\"[LLM] {response['transcript']}\")\n\n# 1. Initialize audio device\npya = pyaudio.PyAudio()\n# 2. Create callback function and session\ncallback = SimpleCallback(pya)\nconv = OmniRealtimeConversation(model=model, callback=callback, url=url)\n# 3. Establish connection and configure session\nconv.connect()\nconv.update_session(output_modalities=[MultiModality.AUDIO, MultiModality.TEXT], voice=voice, instructions=instructions)\n# 4. Initialize audio input stream\nmic = pya.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True)\ncamera = cv2.VideoCapture(0)\nnext_frame_at = 0.0\n# 5. Main loop to process audio and video input\ntry:\n if not camera.isOpened():\n raise RuntimeError(\"Cannot open camera 0.\")\n camera.set(cv2.CAP_PROP_FRAME_WIDTH, video_size[0])\n camera.set(cv2.CAP_PROP_FRAME_HEIGHT, video_size[1])\n camera.set(cv2.CAP_PROP_BUFFERSIZE, 1)\n print(f\"Conversation started with camera ({video_fps} fps, {video_size[0]}x{video_size[1]}), speak into the microphone (Ctrl+C to exit)...\")\n while True:\n audio_data = mic.read(3200, exception_on_overflow=False)\n conv.append_audio(base64.b64encode(audio_data).decode())\n success, frame = camera.read()\n if not success:\n raise RuntimeError(\"Cannot read a camera frame.\")\n if time.monotonic() >= next_frame_at:\n frame = cv2.resize(frame, video_size)\n success, image = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 80])\n if not success:\n raise RuntimeError(\"Cannot encode a camera frame.\")\n conv.append_video(base64.b64encode(image).decode())\n next_frame_at = time.monotonic() + 1 / video_fps\n time.sleep(0.01)\nexcept KeyboardInterrupt:\n pass\nfinally:\n # Clean up resources\n camera.release()\n conv.close()\n mic.close()\n if callback.out:\n callback.out.close()\n pya.terminate()\n print(\"\\nConversation ended\")\n```\n\n### Qwen-Live Harness\n\n[Qwen-Live Harness](https://github.com/QwenLM/Qwen-Live-Harness) is a comprehensive open-source harness designed around the Qwen3.8-Omni-Flash-Realtime API. It can be installed with a single command and integrated into mainstream agent workflows. It supports task delegation, proactive interaction, long-term memory, and context management, and welcomes contributions from the community.\n\n![Qwen-Live Harness interaction framework](https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/Qwen3.8-Omni-Flash/fig3.png?v=20260916-2053#center)\n\nFigure 2. Qwen-Live Harness Interaction Framework.\n\nInstall and get started:\n\n```bash\nnpm install -g qwen-live-harness\nqwen-live-harness init\nqwen-live-harness\n```\n\n## Citation\n\n```bibtex\n@misc{qwen38omniflash,\n title = {Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.},\n url = {https://qwen.ai/blog?id=qwen3.8-omni-flash},\n author = {{Qwen Team}},\n month = {September},\n year = {2026}\n}\n```\n", "authors": [ "QwenTeam" ], "captured_at": "2026-10-19T01:26:53.998049939+00:00", "category": "ai", "content_html": "

\"Qwen3.8-Omni-Flash:

\n

Introduction

\n

Today, we are launching Qwen3.8-Omni-Flash, our next-generation native omnimodal model. Its core objective is to strengthen agent capabilities in real-world productivity scenarios, advancing omnimodal models from “understanding omnimodal content” to “planning tasks, calling tools, and completing creative work.” Building on general agentic capabilities in coding, text-based knowledge work, and GUI operation, Qwen3.8-Omni-Flash further extends agentic applications centered on audio and video, delivering strong results across workflows such as video editing, music video creation, film production and commentary, audio-visual summarization, and real-time conversations.

\n

\"Qwen3.8-Omni-Flash

\n

Figure 1. Qwen3.8-Omni-Flash and its applications in production.

\n
    \n
  • Qwen3.8-Omni-Flash — now available on the Qianwen AI Platform:\n
      \n
    • Text, image, audio, and video inputs with a 1M-token context window.
      • \n
    \n
    • \n
\n

Qwen3.8-Omni-Flash supports a 1M-token context window while maintaining text performance comparable to a text-only model of the same size and delivering significant improvements in omnimodal capabilities. Across 29 evaluations1, its average score improves by more than 25% over Qwen3.5-Omni-Plus; the API price per hour of audio input decreases by more than 98%, and the price per hour of audio-visual input decreases by more than 93%2. For audio-visual agents, coding, and long-horizon tasks, the model improves by 36.5 points on WildClawBench-MM and 22.3 points on AgenticVBench, while scoring a strong 69.6 on UniClawBench. Its core capabilities also improve significantly in long-form audio and audio-visual understanding, audio-visual reasoning, audio-visual captioning, and multi-speaker recognition. For example, it gains 8.3 points on LongAudioSpan and 9.6 points on OmniVideoBench; its OmniCap-IF CSR and ISR improve by 8.5 and 14.1 points, respectively; and its AliMeeting DER and cpWER decrease from 88.11 / 89.61 to 3.35 / 17.18. By scaling data, context, and agentic environments, Qwen3.8-Omni-Flash achieves audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash. These advances also mean that audio and video are evolving from perceptual inputs into core media through which agents understand their environment, reason, and execute tasks.

\n

1. The scope includes audio reasoning benchmarks: AliMeeting-test, AISHELL-4, MagicData-RAMC, MLC-SLM (en), WenetSpeech (Net | Meeting), FLEURS-60 ASR, FLEURS-60 S2TT, SpotSoundBench, MMAU, MMAR, MMSU, MuchoMusic-RUL, HumMusQA, MusTBench, Audio-MultiChallenge, WildSpeech, and VoiceBench; audio-visual reasoning benchmarks: DailyOmni, WorldSense, AVUT, JointAVBench, OmniCloze, OmniCap-IF, QIVD, OmniVideoBench, and StreamingBench; and audio-visual agent benchmarks: WildClawBench-MM, UniClawBench, and OmniGAIA.
\n2. Pricing methodology: hourly audio or audio-visual input prices are estimated as 30 times the input cost of two minutes of source material; audio-visual input uses 720p at 1 fps. Gemini 3.8 Flash uses media_resolution=high, Seed 2.0 Lite uses max_frame_tokens=384, and all other API parameters use their default values. Text input and output prices are in CNY per 1M tokens; Gemini and Muse prices are converted from USD at an exchange rate of 1 USD = 6.7191 CNY.

\n

\"Qwen3.8-Omni-Flash

\n

Audio and video are important media for bringing agents into real-world productivity scenarios, but they also introduce a new set of system-level challenges. Long-form audio and video are costly to store, transmit, and process across multiple rounds of inference; existing agent harness frameworks lack native support for these modalities; and workflows that connect omnimodal understanding with end-to-end task execution are still at an early stage. Addressing these challenges requires models, harness tools, and runtime environments to evolve together.

\n

To address these challenges, we use Qwen3.8-Omni-Flash to explore how to connect source understanding, task planning, tool execution, and result delivery into a complete pipeline. It supports end-to-end, long-horizon workflows such as video editing, translation, film commentary, and content creation, advancing Omni from audio-visual understanding toward autonomous action and task completion.

\n

To this end, we have further expanded Qwen-MM-Plugins with on-demand perception, tool use, and workflow execution for long-form audio and video. We have also open-sourced Qwen-Live Harness as a native runtime for continuous, real-time omnimodal interaction. Together, they address long-horizon workflows and real-time interaction while continuing to expand the capabilities of omnimodal agents alongside the model.

\n

PLUGIN Qwen-MM-Plugins The gateway to audio-visual productivity—connecting multimodal understanding, content creation, and agent harnesses. Get started → HARNESS Qwen-Live Harness The gateway to real-time interaction—connecting audio-visual conversations, task delegation, memory, and context management. Get started →

\n

Long-Form Audio-Visual Understanding

\n

Qwen3.8-Omni-Flash brings a major upgrade to long-form audio-visual understanding—from controllable descriptions and agentic evidence gathering, to understanding meetings and advancing follow-up tasks, and finally to producing video-centered deep research reports. It does not merely process longer content, but finds relevant evidence more precisely, reasons more deeply, and acts more efficiently.

\n

Controllable Audio-Visual Captioning

\n

There is no single answer to how a video should be described. Content creation prioritizes narrative, footage retrieval focuses on specific segments, and asset management depends on structure. Different applications need different video descriptions. In Qwen3.8-Omni-Flash, we have upgraded video captioning from answering "what the model saw" to understanding "what the user wants to know." Users can freely specify the subject, time range, level of detail, and output format. For the same video, the model can provide an overview, locate key segments, or analyze character actions, camera shots, lighting, and sound in depth, producing structured results as needed. You define what to look at, how closely to look, and how to present it.

\n

Agentic Long-Form Audio-Visual Understanding

\n

For videos lasting several hours, conventional approaches require the model to process the entire recording from beginning to end, even when the answer appears in only a few minutes of footage. The native Qwen3.8-Omni-Flash agent starts from the question, independently decides what to watch and listen to, and locates key information through multiple rounds of coarse-to-fine evidence gathering. Without processing every frame, it can focus limited compute and token budgets on the relevant segments, enabling more efficient long-form video understanding. On OmniVideoBench, Agentic Understanding improves accuracy from 63.4 to 67.8 while reducing token consumption from 145,736 to 79,117, a reduction of approximately 45.7%. The table below compares the accuracy and token consumption of Static Understanding and Agentic Understanding on OmniVideoBench:

\n

Note: Agentic mode preserves context across turns.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Static UnderstandingAgentic Understanding
Accuracy (↑)63.467.8
Tokens per query (↓)145,73679,117
\n

Long Meetings: From Minutes to Action

\n

Multi-participant meetings are among the most complex audio-visual understanding scenarios: speakers take turns and overlap, while identities, references, and discussion topics continuously change. Qwen3.8-Omni-Flash jointly recognizes speakers across audio and video and natively supports up to one hour of audio-visual input. It can perform speaker segmentation, content transcription, and identity alignment end to end. Given a complete meeting video and a request, the model can map participant relationships, generate meeting minutes, identify action items, and analyze project risks, using visual information to resolve references and entity ambiguity in the audio. Combined with agents and tool use, it can also send emails, organize tasks, and even begin coding in response to meeting requirements—moving from understanding a meeting to acting on it.

\n

Conducting Deep Research with Audio and Video

\n

When users watch a video with a specific question in mind, the answer often extends beyond the video itself. Qwen3.8-Omni-Flash combines the user’s needs with the video content to identify questions worth deeper investigation, organize the key material, and search multimodal sources across the web—including images, videos, and documents. It then produces a video-centered, richly illustrated research report that helps users understand the content and solve practical problems. For example, when a user encounters color fringing around a Photoshop hair cutout, the model can break down the tutorial steps, study the principles behind Multiply and Screen blend modes, compare alternative edge-repair techniques, and explain which approach best fits the user’s situation.

\n

Qwen3.8-Omni-Flash Conducting Deep Research with Audio and Video Demo

\n

Audio-Visual Production and Editing

\n

Qwen3.8-Omni-Flash is taking audio-visual agents into a new stage: from understanding sounds and images to independently planning, calling tools, and delivering finished videos, bringing omnimodal intelligence into professional audio-visual content production workflows.

\n

Music2MV

\n

For music video (MV) creation, Qwen3.8-Omni-Flash can understand the structure, rhythm, mood, vocals, and instrumental changes of a user-provided song in fine detail, informing the design of characters, scenes, and shots. It can also output line-level lyrics with timestamps to align singing, subtitles, and visuals. Combined with creative tools such as Qwen-MM-Plugins, the model supports the complete workflow from music understanding and creative planning to final quality review, demonstrating strong audio-visual understanding, reasoning, and creation capabilities.

\n

Workflow Next

\n

Final Result 1 Next

\n

Final Result 2 Next

\n

Final Result 3 Next

\n

Short Drama Translation

\n

Traditional video translation often requires repeatedly switching between transcription, translation, dubbing, and editing platforms. This complicates API calls and workflow coordination and makes it difficult to maintain consistency across character voices, dialogue duration, and visual pacing. With an agent built on Qwen3.8-Omni-Flash, users can describe their needs in a single sentence to perform speaker-aware dialogue recognition, conversational translation, character voice cloning and dubbing, audio remixing, and final quality review. These otherwise fragmented localization steps become a complete workflow, enabling automated delivery of short dramas for international audiences.

\n

Workflow Next

\n

Final Result 1 Next

\n

Source

\n

Translated Result

\n

Final Result 2 Next

\n

Source

\n

Translated Result

\n

Final Result 3 Next

\n

Source

\n

Translated Result

\n

Long-Form Film Commentary

\n

Producing commentary videos for full-length films of two or three hours often requires repeatedly watching the film and reconstructing its plot, followed by shot selection, scriptwriting, voiceover, music, and editing. This is a complex and time-consuming process. With an agent built on Qwen3.8-Omni-Flash, users need only provide a film and describe their creative requirements in one sentence to perform long-form video understanding, key-plot extraction, commentary planning, voiceover and music production, editing, rendering, and final quality review. The agent can also intelligently interleave original dialogue with commentary, automatically adjusting speech rate and volume so that narration, original audio, background music, and visuals flow naturally together, creating a more authentic, immersive, and cinematic commentary video.

\n

Workflow Next

\n

Final Result 1 Next

\n

Final Result 2 Next

\n

From Using Models to Optimizing Models

\n

Real-world multimodal applications are often complex and cost-sensitive, requiring models to balance quality, latency, compute, and deployment costs.

\n

Customizing smaller models for specific scenarios is therefore an important path to deploying applications at scale. Yet traditional workflows involve data construction, problem diagnosis, multiple rounds of training, and evaluation, making them time-consuming and heavily dependent on human expertise. This time, we extend Qwen3.8-Omni-Flash into model development itself, exploring a new approach in which large models drive research and development while smaller models serve business needs.

\n

We gave Qwen3.8-Omni-Flash a task: improve Qwen2.5-Omni-3B's Sichuan dialect speech recognition within 12 hours and deliver a usable model. It independently selected the WenetSpeech-Chuan evaluation set, fixed the evaluation criteria, and established a baseline. It then listened directly to audio samples, diagnosed problems using the recognition results, and constructed targeted training data. Across four consecutive rounds of experiments, the agent created 3,413 training examples, adjusted its approach based on evaluation feedback, retained effective improvements, and rolled back unsuccessful attempts. Qwen2.5-Omni-3B's character error rate on the same evaluation set ultimately fell from 25.79% to 15.30%, a relative reduction of approximately 40.7%.

\n

This experiment demonstrates another possibility for model evolution: general-purpose multimodal models understand data, plan experiments, and drive iteration, while smaller models acquire specialized capabilities for specific applications. Agents can go beyond using models to help solve practical business problems.

\n

Audio-Visual Information Compression

\n

Audio and video carry rich information, but their linear, unstructured form makes retrieval and reuse difficult. Qwen3.8-Omni-Flash understands content across sound, visuals, and timelines, using an agentic workflow to extract information, reorganize its structure, and verify results. It transforms the core knowledge and practical experience in long videos into denser information assets that are easier to consume and reuse.

\n

Video2Note

\n

To turn video knowledge into structured resources, we have open-sourced Video2Note in Qwen-MM-Plugins. Drawing on Qwen3.8-Omni-Flash's joint understanding of speech, visuals, and procedures, it automatically organizes knowledge, breaks down key steps, selects representative frames, and generates PDF notes with corresponding text and images. Automated review and iterative correction further condense hours of video into clear, readable documents that are easy to revisit.

\n

Workflow Next

\n

Final Result 1 Next

\n

Example Video

\n

Final PDF

\n

Final Result 2 Next

\n

Example Video

\n

Final PDF

\n

Final Result 3 Next

\n

Example Video

\n

Final PDF

\n

Final Result 4 Next

\n

Example Video

\n

Final PDF

\n

Omni Skill Creator

\n

Videos record not only "how to do something," but also the practical expertise accumulated by specialists. With this in mind, we introduce Omni Skill Creator as a new open-source capability in Qwen-MM-Plugins. It can extract standard operating procedures (SOPs) from demonstrations to perform reusable automated work, or learn tool usage, decision criteria, and key insights from expert instruction. A single demonstration becomes an agent skill that has been verified and evaluated, enabling reusable, shareable skills built from omnimodal content.

\n

Real-Time Audio-Visual Interaction

\n

Qwen3.8-Omni-Flash is designed for deep understanding and creation with complete audio-visual content. For continuous, low-latency interaction, we further introduce Qwen3.8-Omni-Flash-Realtime. It perceives and responds while receiving live audio-visual streams, and uses real-time context to call tools and execute tasks, taking omnimodal capabilities from “understanding a piece of content” to “participating in an interaction.”

\n

Real-Time Speaking Practice

\n

Spoken language has no standard input. Accents, vowel and consonant substitutions, and tonal deviations can cause word-for-word transcription to diverge from the intended meaning. Qwen3.8-Omni-Flash-Realtime jointly models pronunciation and semantics, understands nonstandard expressions affected by accents, aligns them with the correct words, and generates standard-pronunciation demonstrations in real time. Across multiple practice turns, the model updates its judgment with new audio, correcting errors that affect understanding while preserving natural rhythm, tone, and emotion.

\n

Omnimodal Spatial Audio Perception

\n

In real-world spaces, sound provides another coordinate axis beyond vision. Qwen3.8-Omni-Flash-Realtime combines spatial sound with visual information to continuously determine the direction and distance of sound sources while perceiving obstacles, navigable areas, and changes in the scene, making it the first omnimodal model capable of locating targets by sound.

\n

For instructions such as “come over here” or “go see what is making that sound,” the model can isolate voices and target sounds from environmental noise, ground their meaning in the surrounding space, and call tools to perform localization, search, path planning, and navigation—from hearing a target to reaching it.

\n

External Knowledge for Audio-Visual Interaction

\n

Real-time interaction requires not only low latency, but also the ability to load knowledge and behavior dynamically for each application. Qwen3.8-Omni-Flash-Realtime supports injecting identity settings, expression styles, business knowledge, and interaction rules through Skills, while tool use extends these capabilities into task execution.

\n

In scenarios such as customer service, the model can load brand language and service procedures in real time, understand the user's speech, visuals, and context, generate responses that follow business requirements, and execute actions. The same real-time model can therefore take on different knowledge, roles, and ways of acting.

\n

Benchmark Results

\n

Omni

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Qwen3.8-Omni-FlashQwen3.5-Omni-PlusGemini 3.8 FlashSeed 2.0 LiteMuse Spark 1.2
Agentic Omni Intelligence
WildClawBench-MM Multimodal tool use71.034.558.941.9--
UniClawBench Multimodal tool use69.667.169.061.2--
AgenticVBench Multimodal tool use36.814.545.010.0--
OmniGAIA Web Search74.057.278.664.4--
General Audio-Visual Capabilities
DailyOmni Audio-Visual Understanding85.185.184.081.479.6
WorldSense Audio-Visual Understanding68.563.969.667.365.0
AVUT Audio-Visual Understanding86.685.988.081.582.4
JoinAVBench Audio-Visual Understanding75.974.170.470.671.8
OmniVideoBench Audio-Visual Reasoning63.453.865.258.562.2
Video-MME-v2 Audio-Visual Reasoning65.047.971.064.9--
LVOmniBench Long Video Reasoning63.353.270.7----
OmniCloze Audio-Visual Caption63.264.260.956.365.3
OmniCap-IF Audio-Visual CaptionCSR 80.6 ISR 28.2CSR 72.1 ISR 14.1CSR 81.9 ISR 28.3CSR 74.6 ISR 18.1CSR 77.9 ISR 26.8
QIVD Audio-Visual Interaction69.665.669.162.062.0
StreamingBench Audio-Visual Interaction80.857.179.977.277.8
General Audio Capabilities
AliMeeting Test Multi-Speaker ASR (DER | cpWER, ↓)3.4 | 17.288.1 | 89.672.6 | 53.175.1 | 76.193.7 | 92.7
AISHELL-4 Multi-Speaker ASR (DER | cpWER, ↓)2.8 | 11.2100.0 | 100.066.4 | 56.964.8 | 64.291.3 | 86.0
MagicData-RAMC Multi-Speaker ASR (DER | cpWER, ↓)5.7 | 14.198.4 | 97.167.9 | 33.843.4 | 35.182.1 | 75.3
MLC-SLM (en) Multi-Speaker ASR (DER | cpWER, ↓)4.0 | 14.268.6 | 63.960.8 | 26.640.4 | 45.574.3 | 52.9
WenetSpeech (Net) ASR (WER, ↓)4.83.714.24.368.2
WenetSpeech (Meeting) ASR (WER, ↓)4.64.816.74.742.6
FLEURS-ASR Multilingual ASR (WER, ↓)9.37.27.932.123.6
FLEURS-S2TT Multilingual S2TT (BLEU)31.832.233.024.828.8
SpotSoundBench Audio Grounding67.264.239.759.616.9
MMAU Audio Understanding81.881.976.977.263.5
MMAR Audio Understanding79.879.878.577.767.3
MMSU Audio Understanding82.183.083.380.259.9
LongAudioSpan Long Audio ReasoningAccuracy 82.7 Rubric 71.8 Chain 48.2Accuracy 74.4 Rubric 49.8 Chain 45.1Accuracy 79.3 Rubric 65.5 Chain 64.6----
MuchoMusic-RUL Music Understanding72.671.653.761.740.1
HumMusQA Music Understanding75.875.571.266.063.3
MusTBench Music Understanding50.649.140.344.029.4
Audio MultiChallenge Audio Interaction71.557.671.963.457.9
WildSpeech Audio Interaction74.375.776.474.573.4
VoiceBench Audio Interaction91.692.992.384.179.8
\n

1. Evaluation harnesses for Agentic Omni Intelligence: WildClawBench-MM and AgenticVBench use Claude Code, UniClawBench uses OpenClaw, and OmniGAIA uses no harness. For WildClawBench-MM, we evaluate only the multimodal tasks in WildClawBench that involve images, video, or audio.
\n2. FLEURS: ASR and S2TT evaluation results both cover the following 60 languages: Chinese (Mandarin), English, Cantonese, Arabic, German, French, Spanish, Portuguese, Indonesian, Italian, Korean, Russian, Thai, Vietnamese, Japanese, Turkish, Hindi, Malay, Dutch, Urdu, Norwegian, Swedish, Danish, Hebrew, Finnish, Polish, Icelandic, Czech, Filipino, Persian, Greek, Afrikaans, Asturian, Belarusian, Bulgarian, Bengali, Bosnian, Catalan, Cebuano, Estonian, Galician, Gujarati, Croatian, Hungarian, Javanese, Kazakh, Kannada, Kyrgyz, Latvian, Macedonian, Malayalam, Marathi, Punjabi, Romanian, Slovak, Slovenian, Swahili, Tajik, Azerbaijani, and Ukrainian.
\n3. Empty cells (--): scores are not yet available or are not applicable.

\n

Agentic Omni Understanding

\n

Key information in long audio and video recordings is often scattered across different segments, while complex questions require multiple steps of reasoning across sound and images. Agentic Omni Understanding enables the model to start from the question, plan its approach, call tools, and progressively locate and verify evidence. By focusing computation on relevant content, it improves the accuracy and efficiency of long-form audio-visual understanding. To evaluate this capability, we compare Qwen3.8-Omni-Flash and Gemini 3.8 Flash on OmniVideoBench, Video-MME-v2, and LVOmniBench under two settings: Static, where the model directly interprets the input, and an agent mode using Qwen Code. This comparison shows how introducing agent workflows affects each model’s performance.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Qwen3.8-Omni-Flash (Static)Qwen3.8-Omni-Flash (Qwen Code)Gemini 3.8 Flash (Static)Gemini 3.8 Flash (Qwen Code)
OmniVideoBench Audio-Visual Reasoning63.467.865.270.1
Video-MME-v2 Audio-Visual Reasoning65.071.371.072.7
LVOmniBench Long Video Reasoning63.373.670.770.7
\n

Text

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Qwen3.8-Omni-FlashQwen3.8- FlashQwen3.8- 27BQwen3.7- PlusDeepSeek-V4-Flash-0731Claude-Opus-4.6 (Max)
Coding and Agent
DeepSWE 1.1 Long-horizon software engineering57.858.742.216.554.4--
SWE-bench Pro Long-horizon software engineering63.362.561.755.856.053.4
SWE-bench Multilingual Multilingual software engineering80.581.073.875.8--77.5
NL2Repo-Bench Repo-level code generation48.948.142.341.154.247.6
CoWorkBench Long-horizon office work75.373.970.765.145.168.2
General Text Capabilities
IFBench Instruction following81.581.379.579.179.262.5
GPQA Diamond Scientific reasoning91.091.789.290.390.891.3
HLE Multidisciplinary reasoning36.535.930.834.733.840.0
LiveCodeBench v6 Competitive coding92.691.990.389.690.688.8
\n

1. DeepSWE 1.1: evaluated with the Claude Code and mini-SWE-agent harnesses, temp=1.0, top_p=0.95, 256K context window. We report the highest score across the two harnesses; notably, Qwen3.8-Flash-Next performs best on mini-SWE-agent.
\n2. SWE-bench Pro: except for Claude-Opus-4.6 (Max), for which we report the officially published score, all models are evaluated with the Claude Code harness, temp=1.0, top_p=0.95, 256K context window. Problematic tasks were corrected and all baseline models were re-evaluated on the refined benchmark.
\n3. SWE-bench Multilingual: evaluated with the mini-SWE-agent harness, temp=1.0, top_p=0.95, 256K context window.
\n4. NL2Repo-Bench: evaluated with the Claude Code harness. To prevent reward hacking, we disable Bash commands that attempt to access the specific repository, such as pip download, pip install and git clone.
\n5. CoWorkBench: an in-house cowork benchmark for evaluating long-horizon office and productivity agent tasks across computer science, finance, law, medical and other productivity domains.
\n6. HLE: judged by GPT-4o.
\n7. Empty cells (--): scores are not yet available or are not applicable.

\n

Vision

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Qwen3.8-Omni-FlashQwen3.8- FlashQwen3.8- 27BQwen3.7- PlusClaude-Opus-4.6 (Max)
Agentic Vision Intelligence
ClawEval-MM Multimodal tool usePass@3 60.4 Average 61.9Pass@3 64.4 Average 60.4Pass@3 57.4 Average 56.9Pass@3 57.4 Average 60.1Pass@3 52.5 Average 54.7
AndroidWorld Mobile use87.184.581.981.062.0
Vision2Web Visual web development62.964.062.942.1--
General Vision Capabilities
ERQA Embodied intelligence71.072.365.569.840.8
LVBench Long video understanding76.976.672.476.263.0
RealWorldQA Real-world perception87.788.585.986.973.9
MathVision Visual math problem solvingWithout CI 91.8 With CI 96.2Without CI 90.6 With CI 95.7Without CI 90.0 With CI 94.6Without CI 90.3 With CI 88.4Without CI 65.5 With CI --
CharXiv (RQ) Scientific chart analysisWithout CI 83.5 With CI 91.4Without CI 84.6 With CI 90.6Without CI 83.7 With CI 90.2Without CI 85.8 With CI 85.9Without CI 66.0 With CI --
\n

1. ClawEval-MM: scores are reported as "pass@3 / average score". Pass@3 measures the percentage passed in at least one of three trials, and the average score is the mean score across the three trials.
\n2. RecreationBench: an in-house long-horizon application-recreation benchmark for evaluating hybrid-agent abilities spanning five platforms — desktop (Ubuntu, macOS, Windows), mobile (Android) and web.
\n3. OSWorld 2.0: scores are reported as "binary / partial". The binary score is the percentage of tasks that receive the full task reward, while the partial score aggregates the partial rewards obtained across all tasks.
\n4. Vision2Web: scores are reported as the average over the frontend, webpage and website categories, using the Claude Code harness and judged by gpt-5.4-2026-03-05.
\n5. MathVision, CharXiv (RQ): scores are reported as "without CI / with CI". A small number of incorrect ground-truth annotations in MathVision were corrected after manual verification. Our model's score is evaluated using a fixed prompt, e.g. "Please reason step by step, and put your final answer within \\boxed{}." For other models, we report the higher score between runs with and without the \\boxed{} formatting.
\n6. Empty cells (--) indicate scores not yet available or not applicable.

\n

Throughput & Latency

\n

The following results show the observed throughput and latency of the Qwen3.8-Omni-Flash-Realtime API under different input conditions, reflecting the performance users experience in production environments.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Input ScenarioText Output TPS (Tokens/s)Time to First Token (ms)Time to First Audio Packet (ms)Audio Generation RTF
Realtime API Performance
Audio 6s84.87591.26978.360.1538
Audio 12s83.18604.80982.740.1537
Audio 20s81.06617.981026.390.1538
Audio-Visual 6s84.89837.961214.730.1524
Audio-Visual 12s84.33911.851268.070.1527
Audio-Visual 20s83.00981.011350.490.1528
\n

Supported Languages

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
CapabilityLanguagesChinese Dialects
Speech Recognition74 languages: Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Norwegian Nynorsk, Oriya, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajiki, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, and Vietnamese39 dialects: Northeastern Mandarin, Guizhou dialect, Guangdong Cantonese, Henan dialect, Hong Kong Cantonese, Shanghainese, Shaanxi dialect, Tianjin dialect, Taiwanese Mandarin, Yunnan dialect, Anhui dialect, Fujian dialect, Gansu dialect, Guangdong Mandarin, Hubei dialect, Hunan dialect, Jiangxi dialect, Shandong dialect, Shanxi dialect, Sichuanese, Guangxi dialect, Hainan dialect, Chongqing dialect, Changsha dialect, Hangzhou dialect, Hefei dialect, Yinchuan dialect, Zhengzhou dialect, Shenyang dialect, Wenzhou dialect, Wuhan dialect, Kunming dialect, Taiyuan dialect, Nanchang dialect, Jinan dialect, Lanzhou dialect, Nanjing dialect, Hakka, and Southern Min
Speech Generation29 languages: Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Tagalog, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, and Persian7 dialects: Sichuanese, Beijing dialect, Tianjin dialect, Nanjing dialect, Shaanxi dialect, Cantonese, and Southern Min
\n

Getting Started with Qwen3.8-Omni-Flash

\n

API Usage

\n

Qwen3.8-Omni-Flash officially supports reasoning_effort to adjust reasoning depth and control costs:

\n
    \n
  • xhigh (default): for complex tasks that require in-depth analysis.
    • \n
  • medium: balances accuracy and speed.
    • \n
  • low: efficient reasoning optimized for speed and cost.
    • \n
\n

In addition, preserve_thinking is enabled by default across all scenarios for the best out-of-the-box experience.

\n

Qwen3.8-Omni-Flash supports industry-standard protocols, including OpenAI-compatible Chat Completions and Responses APIs. Examples follow:

\n
"""\nEnvironment variables:\n  DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n  DASHSCOPE_BASE_URL: (optional) Base URL for compatible-mode API.\n    - Beijing: https://dashscope.aliyuncs.com/compatible-mode/v1\n    - Singapore: https://dashscope-intl.aliyuncs.com/compatible-mode/v1\n"""\nfrom openai import OpenAI\nimport os\n\napi_key = os.environ.get("DASHSCOPE_API_KEY")\nif not api_key:\n    raise ValueError(\n        "DASHSCOPE_API_KEY is required. "\n        "Set it via: export DASHSCOPE_API_KEY='your-api-key'"\n    )\n\nclient = OpenAI(\n    api_key=api_key,\n    base_url=os.environ.get(\n        "DASHSCOPE_BASE_URL",\n        "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",\n    ),\n)\n\nmessages=[\n    {\n        "role": "user",\n        "content": [\n            {\n                "type": "image_url",\n                "image_url": {\n                    "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"\n                },\n            },\n            {\n                "type": "input_audio",\n                "input_audio": {\n                    "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",\n                    "format": "wav"\n                },\n            },\n            {"type": "text", "text": "Please describe the image and tell me what is being said in the audio."},\n        ],\n    },\n]\n\ncompletion = client.chat.completions.create(\n    model="qwen3.8-omni-flash",\n    messages=messages,\n    extra_body={\n        "enable_thinking": True,\n        # "preserve_thinking": True,\n    },\n    reasoning_effort="xhigh",  # supported levels are xhigh, medium, and low\n    stream=True,\n)\n\nreasoning_content = ""\nanswer_content = ""\nis_answering = False\nprint("\\n" + "=" * 20 + "Reasoning" + "=" * 20 + "\\n")\n\nfor chunk in completion:\n    if not chunk.choices:\n        print("\\nUsage:")\n        print(chunk.usage)\n        continue\n\n    delta = chunk.choices[0].delta\n\n    if hasattr(delta, "reasoning_content") and delta.reasoning_content is not None:\n        if not is_answering:\n            print(delta.reasoning_content, end="", flush=True)\n        reasoning_content += delta.reasoning_content\n\n    if hasattr(delta, "content") and delta.content:\n        if not is_answering:\n            print("\\n" + "=" * 20 + "Answer" + "=" * 20 + "\\n")\n            is_answering = True\n        print(delta.content, end="", flush=True)\n        answer_content += delta.content\n
\n

Best Practices for Controllable Audio-Visual Captioning

\n

Content organization and level of detail: Organize video descriptions, OCR extraction, audio descriptions, and speech transcripts into sections covering visuals, speech, music, sound effects, and ambient sounds. Preserve the original text, speaker identities, and corresponding time ranges to make retrieval and verification easier. Specify a target length in the prompt to control detail, for example, Describe the video in approximately 2000–3000 words., and adjust it to the video’s duration and information density.

\n

Prompt Example

\n
Provide a detailed description of the video.\n\nMake sure your description covers every one of the following dimensions:\n\nVisual\n- Subjects and characters: appearance, clothing, gender/age cues, identity, distinctive features\n- Actions and events in chronological order, and how the scene evolves over time\n- Setting and background: location, environment, time of day\n- Spatial layout and relations between subjects/objects; counts and quantities\n- On-screen text: captions, titles, subtitles, logos, UI — exact content and appearance\n- Visual style: colors, lighting, camera shots, angles, and camera movement\n\nAudio\n- Speech: the exact spoken content, transcribed verbatim\n- Speakers: who is speaking (mapped to the on-screen person or voice-over), with accent, tone, gender/age cues\n- Speaking state: prosody, emotion, volume, and speaking style\n- Music: presence, genre/mood, and lyrics if any\n- Sound effects and ambient/background sounds\n- Non-speech vocalizations: laughter, crying, applause, etc.\n\nAudio-visual correspondence\n- Which speech or sound aligns with which on-screen person or visual event\n- The timing of each event, expressed with timestamps\n\nIt should explicitly include three sections:\n\n1. A structured chronological storyline of **every noticeable audio and visual details**\n2. A structured list of all visible text. For each text element, include start timestamp, end timestamp, the exact text content, the appearance characteristics. If no text appears, explicitly state so.\n3. A structured speech-to-text transcription, include speaker (corresponding to the character or voice-over in Section 1, including their accent and tone), exact spoken content, start timestamp, end timestamp, and speaking state (prosody, emotion, and style). If no speech appears, explicitly state so.\n\nAside from these three required sections, you are free to organize any additional content in any way you find helpful. This additional content can include global information about the entire video or localized information about specific moments. You may choose the topic of this extra content freely.\n\nRules:\n\n- Add as much descriptive detail as possible.\n- Do not use Markdown bold formatting.\n- Carefully look at frames and listen to the audio, making sure no detail is overlooked.\n\nOutput Format:\n```\n## Storyline\n\n<xx:xx.xxx> - <xx:xx.xxx>\n<an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>\n\n<xx:xx.xxx> - <xx:xx.xxx>\n<an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>\n\n<xx:xx.xxx> - <xx:xx.xxx>\n<an unstructured long paragraph in natural language describing what happened during this period, blending both audio and video details.>\n\n...\n\n## Visible Text\n\n<xx:xx.xxx> - <xx:xx.xxx>\n“<element>”: <appearance>\n“<element>”: <appearance>\n\n<xx:xx.xxx> - <xx:xx.xxx>\n“<element>”: <appearance>\n“<element>”: <appearance>\n“<element>”: <appearance>\n\n<xx:xx.xxx> - <xx:xx.xxx>\n“<element>”: <appearance>\n\n...\n\n## Speakers and Transcript\n\nSpeaker profiles:\n<speaker> - <profile>\n<speaker> - <profile>\n<speaker> - <profile>\n...\n\n<xx:xx.xxx> - <xx:xx.xxx>\nSpeaker: <speaker>\nState: <description>\nContent: “<content>”\n\n<xx:xx.xxx> - <xx:xx.xxx>\nSpeaker: <speaker>\nState: <description>\nContent: “<content>”\n\n<xx:xx.xxx> - <xx:xx.xxx>\nSpeaker: <speaker>\nState: <description>\nContent: “<content>”\n\n...\n\n## <another section>\n\n<paragraphs>\n\n## <another section>\n\n<paragraphs>\n\n...\n```\n
\n

Structured output and schema constraints: Include task instructions and a complete JSON Schema in the prompt, specifying field meanings, types, required fields, and constraints to support programmatic parsing and downstream use.

\n

Prompt Example

\n
Describe the audio and visual content in detail in English, organized into scenes and events, following the JSON Schema below.\nAll timestamps must be relative to the beginning of the video. End times must not precede start times or exceed the video duration. Each event must fall within the time range of its parent scene.\nInclude only information directly supported by the audio or video. Do not guess or invent details. Do not infer causality merely because a sound and an action occur at the same time.\nReturn only valid JSON, without Markdown fences or commentary.\n\nJSON Schema:\n{\n  "$defs": {\n    "Event": {\n      "additionalProperties": false,\n      "properties": {\n        "time_range": {\n          "$ref": "#/$defs/TimeRange",\n          "description": "Time range of the event"\n        },\n        "participants": {\n          "description": "People, animals, or objects involved, named by observable features; use consistent names for the same participant",\n          "items": {\n            "type": "string"\n          },\n          "title": "Participants",\n          "type": "array"\n        },\n        "action": {\n          "description": "Specific actions, interactions, and observable outcomes",\n          "title": "Action",\n          "type": "string"\n        },\n        "sounds": {\n          "description": "Sounds heard during the event; use an empty list if none are discernible",\n          "items": {\n            "type": "string"\n          },\n          "title": "Sounds",\n          "type": "array"\n        }\n      },\n      "required": [\n        "time_range",\n        "participants",\n        "action",\n        "sounds"\n      ],\n      "title": "Event",\n      "type": "object"\n    },\n    "Scene": {\n      "additionalProperties": false,\n      "properties": {\n        "time_range": {\n          "$ref": "#/$defs/TimeRange",\n          "description": "Time range of the scene"\n        },\n        "setting": {\n          "description": "Environment, spatial layout, and main visual features",\n          "title": "Setting",\n          "type": "string"\n        },\n        "events": {\n          "description": "Events in chronological order; use an empty list if there are none",\n          "items": {\n            "$ref": "#/$defs/Event"\n          },\n          "title": "Events",\n          "type": "array"\n        }\n      },\n      "required": [\n        "time_range",\n        "setting",\n        "events"\n      ],\n      "title": "Scene",\n      "type": "object"\n    },\n    "TimeRange": {\n      "additionalProperties": false,\n      "properties": {\n        "start_seconds": {\n          "description": "Start time in seconds relative to the beginning of the video",\n          "minimum": 0,\n          "title": "Start Seconds",\n          "type": "number"\n        },\n        "end_seconds": {\n          "description": "End time in seconds; must not precede the start time",\n          "minimum": 0,\n          "title": "End Seconds",\n          "type": "number"\n        }\n      },\n      "required": [\n        "start_seconds",\n        "end_seconds"\n      ],\n      "title": "TimeRange",\n      "type": "object"\n    }\n  },\n  "additionalProperties": false,\n  "properties": {\n    "summary": {\n      "description": "An overview of the main content of the video",\n      "title": "Summary",\n      "type": "string"\n    },\n    "scenes": {\n      "description": "Scenes in chronological order; group continuous footage with a consistent setting into one scene",\n      "items": {\n        "$ref": "#/$defs/Scene"\n      },\n      "title": "Scenes",\n      "type": "array"\n    }\n  },\n  "required": [\n    "summary",\n    "scenes"\n  ],\n  "title": "CaptionResult",\n  "type": "object"\n}\n
\n

Installing Qwen-MM-Plugins in an Agent Harness

\n

Qwen-MM-Plugins is a multimodal plugin suite for agent harnesses. It gives agents the ability to understand images, audio, video, and documents, as well as maintain memory for long-form video and create content. It supports agent harnesses including Codex, Claude Code, Qwen Code, Gemini CLI, Qoder, CodeBuddy, and OpenClaw. We also welcome contributions from community developers.

\n

You can enter the following request directly in your usual office agent:

\n
Help me install the core, api, and omni-related plugins from https://github.com/QwenLM/Qwen-MM-Plugins.\n
\n

\"Installing

\n

Alternatively, install from the command line:

\n
# Run the official guided installer:\ncurl -fsSL https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh | bash\n
\n

In the menu, select:

\n
    \n
  • Install
    • \n
  • The agent harness you use
    • \n
  • The Omni plugins you need
    • \n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PluginCapability
coreRead images, video frames, PDFs, Office documents, code, data, and 3D files
apiImage understanding, OCR, object localization, audio-visual transcription, speaker diarization, event analysis, and image segmentation
omni-chatcutCreate MVs, long-form film commentary, and video speech translations
omni-video2noteTurn video tutorials into PDF notes with key screenshots
omni-skill-creatorTurn instructional videos, screen recordings, or operation demonstrations into reusable Agent Skill.md files
omni-memoryBuild memory for people, dialogue, sounds, and events in long-form videos
\n

After installation, restart the agent harness or create a new task.

\n
@song.mp3 Generate a complete MV based on the song's rhythm and content.\n@short_drama.mp4 Translate the video into English while preserving the original speakers' voice characteristics where possible.\n@movie.mp4 Create a film commentary video with Chinese narration and subtitles.\n\n@weekly_report_sop.mp4 Turn this screen recording of writing a weekly report into a Skill.md file.\n@tutorial.mp4 Turn the tutorial into PDF notes with key screenshots, timestamps, and step-by-step instructions.\n@documentary.mp4 Build audio-visual memory that records people, dialogue, sounds, and important events.\n
\n

Getting Started with Qwen3.8-Omni-Flash-Realtime

\n

API Usage

\n

Qwen3.8-Omni-Flash-Realtime supports connections over WebSocket and WebRTC. Running the basic examples below opens the camera and microphone for a real-time audio-visual conversation. We recommend using headphones.

\n
# Run pip install websocket-client pyaudio dashscope opencv-python -U to install dependencies\n"""\nEnvironment variables:\n  DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n  DASHSCOPE_BASE_URL: (optional) Base URL for realtime API.\n    - Beijing: wss://dashscope.aliyuncs.com/api-ws/v1/realtime\n    - Singapore: wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime\n"""\nimport os\nimport base64\nimport time\nimport pyaudio\nimport cv2\nfrom dashscope.audio.qwen_omni import MultiModality, AudioFormat,OmniRealtimeCallback,OmniRealtimeConversation\nimport dashscope\n\nurl = f'wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime'\ndashscope.api_key = os.getenv('DASHSCOPE_API_KEY')\n# Determine the voice\nvoice = 'Tina'\n# Determine the model\nmodel = 'qwen3.8-omni-flash-realtime'\n# Determine the model role\ninstructions = "You are Qwen-Omni, a helpful assistant."\nvideo_fps = 1\nvideo_size = (1280, 720)\nclass SimpleCallback(OmniRealtimeCallback):\n    def __init__(self, pya):\n        self.pya = pya\n        self.out = None\n    def on_open(self):\n        # Initialize audio output stream\n        self.out = self.pya.open(\n            format=pyaudio.paInt16,\n            channels=1,\n            rate=24000,\n            output=True\n        )\n    def on_event(self, response):\n        if response['type'] == 'response.audio.delta':\n            # Play audio\n            self.out.write(base64.b64decode(response['delta']))\n        elif response['type'] == 'conversation.item.input_audio_transcription.delta':\n            # Streaming preview: text is the confirmed prefix, stash is the confirmed suffix\n            preview = response.get('text', '') + response.get('stash', '')\n            print(f"\\r[User] {preview}", end='', flush=True)\n        elif response['type'] == 'conversation.item.input_audio_transcription.completed':\n            # Transcription completed, print the final text and a new line\n            print(f"\\r[User] {response['transcript']}")\n        elif response['type'] == 'response.audio_transcript.done':\n            # Print the assistant's response text\n            print(f"[LLM] {response['transcript']}")\n\n# 1. Initialize audio device\npya = pyaudio.PyAudio()\n# 2. Create callback function and session\ncallback = SimpleCallback(pya)\nconv = OmniRealtimeConversation(model=model, callback=callback, url=url)\n# 3. Establish connection and configure session\nconv.connect()\nconv.update_session(output_modalities=[MultiModality.AUDIO, MultiModality.TEXT], voice=voice, instructions=instructions)\n# 4. Initialize audio input stream\nmic = pya.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True)\ncamera = cv2.VideoCapture(0)\nnext_frame_at = 0.0\n# 5. Main loop to process audio and video input\ntry:\n    if not camera.isOpened():\n        raise RuntimeError("Cannot open camera 0.")\n    camera.set(cv2.CAP_PROP_FRAME_WIDTH, video_size[0])\n    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, video_size[1])\n    camera.set(cv2.CAP_PROP_BUFFERSIZE, 1)\n    print(f"Conversation started with camera ({video_fps} fps, {video_size[0]}x{video_size[1]}), speak into the microphone (Ctrl+C to exit)...")\n    while True:\n        audio_data = mic.read(3200, exception_on_overflow=False)\n        conv.append_audio(base64.b64encode(audio_data).decode())\n        success, frame = camera.read()\n        if not success:\n            raise RuntimeError("Cannot read a camera frame.")\n        if time.monotonic() >= next_frame_at:\n            frame = cv2.resize(frame, video_size)\n            success, image = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 80])\n            if not success:\n                raise RuntimeError("Cannot encode a camera frame.")\n            conv.append_video(base64.b64encode(image).decode())\n            next_frame_at = time.monotonic() + 1 / video_fps\n        time.sleep(0.01)\nexcept KeyboardInterrupt:\n    pass\nfinally:\n    # Clean up resources\n    camera.release()\n    conv.close()\n    mic.close()\n    if callback.out:\n        callback.out.close()\n    pya.terminate()\n    print("\\nConversation ended")\n
\n

Qwen-Live Harness

\n

Qwen-Live Harness is a comprehensive open-source harness designed around the Qwen3.8-Omni-Flash-Realtime API. It can be installed with a single command and integrated into mainstream agent workflows. It supports task delegation, proactive interaction, long-term memory, and context management, and welcomes contributions from the community.

\n

\"Qwen-Live

\n

Figure 2. Qwen-Live Harness Interaction Framework.

\n

Install and get started:

\n
npm install -g qwen-live-harness\nqwen-live-harness init\nqwen-live-harness\n
\n

Citation

\n
@misc{qwen38omniflash,\n    title = {Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.},\n    url = {https://qwen.ai/blog?id=qwen3.8-omni-flash},\n    author = {{Qwen Team}},\n    month = {September},\n    year = {2026}\n}\n
\n", "content_markdown": "![Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/banner-latest.en.png)\n\n## Introduction\n\nToday, we are launching **[Qwen3.8-Omni-Flash](https://www.alibabacloud.com/help/en/model-studio/qwen-omni)**, our next-generation native omnimodal model. Its core objective is to strengthen agent capabilities in real-world productivity scenarios, advancing omnimodal models from “understanding omnimodal content” to “planning tasks, calling tools, and completing creative work.” Building on general agentic capabilities in coding, text-based knowledge work, and GUI operation, Qwen3.8-Omni-Flash further extends agentic applications centered on audio and video, delivering strong results across workflows such as video editing, music video creation, film production and commentary, audio-visual summarization, and real-time conversations.\n\n![Qwen3.8-Omni-Flash and its applications in production](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/production-overview-v2.png#center)\n\nFigure 1. Qwen3.8-Omni-Flash and its applications in production.\n\n- **Qwen3.8-Omni-Flash** — now available on the [Qianwen AI Platform](https://www.qianwenai.com/):\n - Text, image, audio, and video inputs with a **1M-token context window**.\n\nQwen3.8-Omni-Flash supports a 1M-token context window while maintaining text performance comparable to a text-only model of the same size and delivering significant improvements in omnimodal capabilities. Across 29 evaluations1, its average score improves by more than 25% over Qwen3.5-Omni-Plus; the API price per hour of audio input decreases by more than 98%, and the price per hour of audio-visual input decreases by more than 93%2. For audio-visual agents, coding, and long-horizon tasks, the model improves by 36.5 points on WildClawBench-MM and 22.3 points on AgenticVBench, while scoring a strong 69.6 on UniClawBench. Its core capabilities also improve significantly in long-form audio and audio-visual understanding, audio-visual reasoning, audio-visual captioning, and multi-speaker recognition. For example, it gains 8.3 points on LongAudioSpan and 9.6 points on OmniVideoBench; its OmniCap-IF CSR and ISR improve by 8.5 and 14.1 points, respectively; and its AliMeeting DER and cpWER decrease from 88.11 / 89.61 to 3.35 / 17.18. By scaling data, context, and agentic environments, Qwen3.8-Omni-Flash achieves **audio-visual performance close to Gemini 3.8 Flash and overall audio performance that exceeds Gemini 3.8 Flash**. These advances also mean that audio and video are evolving from perceptual inputs into core media through which agents understand their environment, reason, and execute tasks.\n\n1\\. The scope includes audio reasoning benchmarks: AliMeeting-test, AISHELL-4, MagicData-RAMC, MLC-SLM (en), WenetSpeech (Net | Meeting), FLEURS-60 ASR, FLEURS-60 S2TT, SpotSoundBench, MMAU, MMAR, MMSU, MuchoMusic-RUL, HumMusQA, MusTBench, Audio-MultiChallenge, WildSpeech, and VoiceBench; audio-visual reasoning benchmarks: DailyOmni, WorldSense, AVUT, JointAVBench, OmniCloze, OmniCap-IF, QIVD, OmniVideoBench, and StreamingBench; and audio-visual agent benchmarks: WildClawBench-MM, UniClawBench, and OmniGAIA.\\\n2\\. Pricing methodology: hourly audio or audio-visual input prices are estimated as 30 times the input cost of two minutes of source material; audio-visual input uses 720p at 1 fps. Gemini 3.8 Flash uses media\\_resolution=high, Seed 2.0 Lite uses max\\_frame\\_tokens=384, and all other API parameters use their default values. Text input and output prices are in CNY per 1M tokens; Gemini and Muse prices are converted from USD at an exchange rate of 1 USD = 6.7191 CNY.\n\n![Qwen3.8-Omni-Flash pricing and representative benchmark comparison](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/performance-overview-v3.en.png)\n\nAudio and video are important media for bringing agents into real-world productivity scenarios, but they also introduce a new set of system-level challenges. Long-form audio and video are costly to store, transmit, and process across multiple rounds of inference; existing agent harness frameworks lack native support for these modalities; and workflows that connect omnimodal understanding with end-to-end task execution are still at an early stage. Addressing these challenges requires models, harness tools, and runtime environments to evolve together.\n\nTo address these challenges, we use Qwen3.8-Omni-Flash to explore how to connect source understanding, task planning, tool execution, and result delivery into a complete pipeline. It supports end-to-end, long-horizon workflows such as video editing, translation, film commentary, and content creation, advancing Omni from audio-visual understanding toward autonomous action and task completion.\n\nTo this end, we have further expanded Qwen-MM-Plugins with on-demand perception, tool use, and workflow execution for long-form audio and video. We have also open-sourced Qwen-Live Harness as a native runtime for continuous, real-time omnimodal interaction. Together, they address long-horizon workflows and real-time interaction while continuing to expand the capabilities of omnimodal agents alongside the model.\n\n[PLUGIN Qwen-MM-Plugins The gateway to audio-visual productivity—connecting multimodal understanding, content creation, and agent harnesses. Get started →](https://qwen.ai/blog?id=qwen3.8-omni-flash#IbE99) [HARNESS Qwen-Live Harness The gateway to real-time interaction—connecting audio-visual conversations, task delegation, memory, and context management. Get started →](https://qwen.ai/blog?id=qwen3.8-omni-flash#iTpXF)\n\n## Long-Form Audio-Visual Understanding\n\nQwen3.8-Omni-Flash brings a major upgrade to long-form audio-visual understanding—from controllable descriptions and agentic evidence gathering, to understanding meetings and advancing follow-up tasks, and finally to producing video-centered deep research reports. It does not merely process longer content, but finds relevant evidence more precisely, reasons more deeply, and acts more efficiently.\n\n### Controllable Audio-Visual Captioning\n\nThere is no single answer to how a video should be described. Content creation prioritizes narrative, footage retrieval focuses on specific segments, and asset management depends on structure. Different applications need different video descriptions. In Qwen3.8-Omni-Flash, we have upgraded video captioning from answering \"what the model saw\" to understanding \"what the user wants to know.\" Users can freely specify the subject, time range, level of detail, and output format. For the same video, the model can provide an overview, locate key segments, or **analyze character actions, camera shots, lighting, and sound in depth, producing structured results as needed**. You define what to look at, how closely to look, and how to present it.\n\n### Agentic Long-Form Audio-Visual Understanding\n\nFor videos lasting several hours, conventional approaches require the model to process the entire recording from beginning to end, even when the answer appears in only a few minutes of footage. The native Qwen3.8-Omni-Flash agent starts from the question, **independently decides what to watch and listen to, and locates key information through multiple rounds of coarse-to-fine evidence gathering**. Without processing every frame, it can focus limited compute and token budgets on the relevant segments, enabling more efficient long-form video understanding. On OmniVideoBench, Agentic Understanding **improves accuracy from 63.4 to 67.8** while reducing token consumption from 145,736 to 79,117, **a reduction of approximately 45.7%**. The table below compares the accuracy and token consumption of Static Understanding and Agentic Understanding on OmniVideoBench:\n\nNote: Agentic mode preserves context across turns.\n| | Static Understanding | Agentic Understanding |\n| -------------------- | -------------------- | --------------------- |\n| Accuracy (↑) | 63.4 | 67.8 |\n| Tokens per query (↓) | 145,736 | 79,117 |\n\n### Long Meetings: From Minutes to Action\n\nMulti-participant meetings are among the most complex audio-visual understanding scenarios: speakers take turns and overlap, while identities, references, and discussion topics continuously change. Qwen3.8-Omni-Flash **jointly recognizes speakers across audio and video** and natively supports **up to one hour** of audio-visual input. It can **perform speaker segmentation, content transcription, and identity alignment end to end**. Given a complete meeting video and a request, the model can map participant relationships, generate meeting minutes, identify action items, and analyze project risks, using visual information to resolve references and entity ambiguity in the audio. Combined with agents and tool use, it can also send emails, organize tasks, and even begin coding in response to meeting requirements—moving from understanding a meeting to acting on it.\n\n### Conducting Deep Research with Audio and Video\n\nWhen users watch a video with a specific question in mind, the answer often extends beyond the video itself. Qwen3.8-Omni-Flash combines the user’s needs with the video content to identify questions worth deeper investigation, organize the key material, and search multimodal sources across the web—including images, videos, and documents. It then produces a video-centered, richly illustrated research report that helps users understand the content and solve practical problems. For example, when a user encounters color fringing around a Photoshop hair cutout, the model can break down the tutorial steps, study the principles behind Multiply and Screen blend modes, compare alternative edge-repair techniques, and explain which approach best fits the user’s situation.\n\n[Qwen3.8-Omni-Flash Conducting Deep Research with Audio and Video Demo](https://docs.qwenlm.ai/resources/qwen3.8-omni-flash/omnidr-demo-0813-0600.51939eff.html)\n\n## Audio-Visual Production and Editing\n\nQwen3.8-Omni-Flash is taking audio-visual agents into a new stage: **from understanding sounds and images to independently planning, calling tools, and delivering finished videos**, bringing omnimodal intelligence into professional audio-visual content production workflows.\n\n### Music2MV\n\nFor music video (MV) creation, Qwen3.8-Omni-Flash can understand the structure, rhythm, mood, vocals, and instrumental changes of a user-provided song in fine detail, informing the design of characters, scenes, and shots. It can also output line-level lyrics with timestamps to align singing, subtitles, and visuals. Combined with creative tools such as Qwen-MM-Plugins, the model supports the complete workflow from music understanding and creative planning to final quality review, demonstrating strong audio-visual understanding, reasoning, and creation capabilities.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nFinal Result 2 Next\n\nFinal Result 3 Next\n\n### Short Drama Translation\n\nTraditional video translation often requires repeatedly switching between transcription, translation, dubbing, and editing platforms. This complicates API calls and workflow coordination and makes it difficult to maintain consistency across character voices, dialogue duration, and visual pacing. With an agent built on Qwen3.8-Omni-Flash, users can describe their needs in a single sentence to perform speaker-aware dialogue recognition, conversational translation, character voice cloning and dubbing, audio remixing, and final quality review. These otherwise fragmented localization steps become a complete workflow, enabling automated delivery of short dramas for international audiences.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nSource\n\nTranslated Result\n\nFinal Result 2 Next\n\nSource\n\nTranslated Result\n\nFinal Result 3 Next\n\nSource\n\nTranslated Result\n\n### Long-Form Film Commentary\n\nProducing commentary videos for full-length films of two or three hours often requires repeatedly watching the film and reconstructing its plot, followed by shot selection, scriptwriting, voiceover, music, and editing. This is a complex and time-consuming process. With an agent built on Qwen3.8-Omni-Flash, users need only provide a film and describe their creative requirements in one sentence to perform long-form video understanding, key-plot extraction, commentary planning, voiceover and music production, editing, rendering, and final quality review. The agent can also intelligently interleave original dialogue with commentary, automatically adjusting speech rate and volume so that narration, original audio, background music, and visuals flow naturally together, creating a more authentic, immersive, and cinematic commentary video.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nFinal Result 2 Next\n\n## From Using Models to Optimizing Models\n\nReal-world multimodal applications are often complex and cost-sensitive, requiring models to balance quality, latency, compute, and deployment costs.\n\nCustomizing smaller models for specific scenarios is therefore an important path to deploying applications at scale. Yet traditional workflows involve data construction, problem diagnosis, multiple rounds of training, and evaluation, making them time-consuming and heavily dependent on human expertise. This time, we extend Qwen3.8-Omni-Flash into model development itself, exploring a new approach in which large models drive research and development while smaller models serve business needs.\n\nWe gave Qwen3.8-Omni-Flash a task: **improve Qwen2.5-Omni-3B's Sichuan dialect speech recognition within 12 hours and deliver a usable model**. It independently selected the WenetSpeech-Chuan evaluation set, fixed the evaluation criteria, and established a baseline. It then listened directly to audio samples, diagnosed problems using the recognition results, and constructed targeted training data. Across four consecutive rounds of experiments, the agent created 3,413 training examples, adjusted its approach based on evaluation feedback, retained effective improvements, and rolled back unsuccessful attempts. **Qwen2.5-Omni-3B's character error rate on the same evaluation set ultimately fell from 25.79% to 15.30%, a relative reduction of approximately 40.7%**.\n\nThis experiment demonstrates another possibility for model evolution: general-purpose multimodal models understand data, plan experiments, and drive iteration, while smaller models acquire specialized capabilities for specific applications. Agents can go beyond using models to help solve practical business problems.\n\n## Audio-Visual Information Compression\n\nAudio and video carry rich information, but their linear, unstructured form makes retrieval and reuse difficult. Qwen3.8-Omni-Flash understands content across sound, visuals, and timelines, using an agentic workflow to extract information, reorganize its structure, and verify results. **It transforms the core knowledge and practical experience in long videos into denser information assets that are easier to consume and reuse**.\n\n### Video2Note\n\nTo turn video knowledge into structured resources, we have open-sourced Video2Note in Qwen-MM-Plugins. Drawing on Qwen3.8-Omni-Flash's joint understanding of speech, visuals, and procedures, it automatically organizes knowledge, breaks down key steps, selects representative frames, and generates PDF notes with corresponding text and images. Automated review and iterative correction further condense hours of video into clear, readable documents that are easy to revisit.\n\nWorkflow Next\n\nFinal Result 1 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 2 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 3 Next\n\nExample Video\n\nFinal PDF\n\nFinal Result 4 Next\n\nExample Video\n\nFinal PDF\n\n### Omni Skill Creator\n\nVideos record not only \"how to do something,\" but also the practical expertise accumulated by specialists. With this in mind, we introduce Omni Skill Creator as a new open-source capability in Qwen-MM-Plugins. It can extract standard operating procedures (SOPs) from demonstrations to perform reusable automated work, or learn tool usage, decision criteria, and key insights from expert instruction. A single demonstration becomes an agent skill that has been verified and evaluated, enabling reusable, shareable skills built from omnimodal content.\n\n## Real-Time Audio-Visual Interaction\n\nQwen3.8-Omni-Flash is designed for deep understanding and creation with complete audio-visual content. For continuous, low-latency interaction, we further introduce **Qwen3.8-Omni-Flash-Realtime**. It perceives and responds while receiving live audio-visual streams, and uses real-time context to call tools and execute tasks, taking omnimodal capabilities from “understanding a piece of content” to “participating in an interaction.”\n\n### Real-Time Speaking Practice\n\nSpoken language has no standard input. Accents, vowel and consonant substitutions, and tonal deviations can cause word-for-word transcription to diverge from the intended meaning. Qwen3.8-Omni-Flash-Realtime jointly models pronunciation and semantics, understands nonstandard expressions affected by accents, aligns them with the correct words, and generates standard-pronunciation demonstrations in real time. Across multiple practice turns, the model updates its judgment with new audio, correcting errors that affect understanding while preserving natural rhythm, tone, and emotion.\n\n### Omnimodal Spatial Audio Perception\n\nIn real-world spaces, sound provides another coordinate axis beyond vision. Qwen3.8-Omni-Flash-Realtime combines spatial sound with visual information to continuously determine the direction and distance of sound sources while perceiving obstacles, navigable areas, and changes in the scene, making it the first omnimodal model capable of locating targets by sound.\n\nFor instructions such as “come over here” or “go see what is making that sound,” the model can isolate voices and target sounds from environmental noise, ground their meaning in the surrounding space, and call tools to perform localization, search, path planning, and navigation—from hearing a target to reaching it.\n\n### External Knowledge for Audio-Visual Interaction\n\nReal-time interaction requires not only low latency, but also the ability to load knowledge and behavior dynamically for each application. Qwen3.8-Omni-Flash-Realtime supports injecting identity settings, expression styles, business knowledge, and interaction rules through Skills, while tool use extends these capabilities into task execution.\n\nIn scenarios such as customer service, the model can load brand language and service procedures in real time, understand the user's speech, visuals, and context, generate responses that follow business requirements, and execute actions. The same real-time model can therefore take on different knowledge, roles, and ways of acting.\n\n## Benchmark Results\n\n### Omni\n\n| | Qwen3.8-Omni-Flash | Qwen3.5-Omni-Plus | Gemini 3.8 Flash | Seed 2.0 Lite | Muse Spark 1.2 |\n| --------------------------------------------------- | ------------------------------------------------- | ----------------------------------------- | --------------------------------------------- | -------------------- | -------------------- |\n| Agentic Omni Intelligence | | | | | |\n| WildClawBench-MM Multimodal tool use | **71.0** | 34.5 | 58.9 | 41.9 | -- |\n| UniClawBench Multimodal tool use | **69.6** | 67.1 | 69.0 | 61.2 | -- |\n| AgenticVBench Multimodal tool use | 36.8 | 14.5 | **45.0** | 10.0 | -- |\n| OmniGAIA Web Search | 74.0 | 57.2 | **78.6** | 64.4 | -- |\n| General Audio-Visual Capabilities | | | | | |\n| DailyOmni Audio-Visual Understanding | **85.1** | **85.1** | 84.0 | 81.4 | 79.6 |\n| WorldSense Audio-Visual Understanding | 68.5 | 63.9 | **69.6** | 67.3 | 65.0 |\n| AVUT Audio-Visual Understanding | 86.6 | 85.9 | **88.0** | 81.5 | 82.4 |\n| JoinAVBench Audio-Visual Understanding | **75.9** | 74.1 | 70.4 | 70.6 | 71.8 |\n| OmniVideoBench Audio-Visual Reasoning | 63.4 | 53.8 | **65.2** | 58.5 | 62.2 |\n| Video-MME-v2 Audio-Visual Reasoning | 65.0 | 47.9 | **71.0** | 64.9 | -- |\n| LVOmniBench Long Video Reasoning | 63.3 | 53.2 | **70.7** | -- | -- |\n| OmniCloze Audio-Visual Caption | 63.2 | 64.2 | 60.9 | 56.3 | **65.3** |\n| OmniCap-IF Audio-Visual Caption | CSR 80.6 ISR 28.2 | CSR 72.1 ISR 14.1 | CSR **81.9** ISR **28.3** | CSR 74.6 ISR 18.1 | CSR 77.9 ISR 26.8 |\n| QIVD Audio-Visual Interaction | **69.6** | 65.6 | 69.1 | 62.0 | 62.0 |\n| StreamingBench Audio-Visual Interaction | **80.8** | 57.1 | 79.9 | 77.2 | 77.8 |\n| General Audio Capabilities | | | | | |\n| AliMeeting Test Multi-Speaker ASR (DER | cpWER, ↓) | **3.4** | **17.2** | 88.1 | 89.6 | 72.6 | 53.1 | 75.1 | 76.1 | 93.7 | 92.7 |\n| AISHELL-4 Multi-Speaker ASR (DER | cpWER, ↓) | **2.8** | **11.2** | 100.0 | 100.0 | 66.4 | 56.9 | 64.8 | 64.2 | 91.3 | 86.0 |\n| MagicData-RAMC Multi-Speaker ASR (DER | cpWER, ↓) | **5.7** | **14.1** | 98.4 | 97.1 | 67.9 | 33.8 | 43.4 | 35.1 | 82.1 | 75.3 |\n| MLC-SLM (en) Multi-Speaker ASR (DER | cpWER, ↓) | **4.0** | **14.2** | 68.6 | 63.9 | 60.8 | 26.6 | 40.4 | 45.5 | 74.3 | 52.9 |\n| WenetSpeech (Net) ASR (WER, ↓) | 4.8 | **3.7** | 14.2 | 4.3 | 68.2 |\n| WenetSpeech (Meeting) ASR (WER, ↓) | **4.6** | 4.8 | 16.7 | 4.7 | 42.6 |\n| FLEURS-ASR Multilingual ASR (WER, ↓) | 9.3 | **7.2** | 7.9 | 32.1 | 23.6 |\n| FLEURS-S2TT Multilingual S2TT (BLEU) | 31.8 | 32.2 | **33.0** | 24.8 | 28.8 |\n| SpotSoundBench Audio Grounding | **67.2** | 64.2 | 39.7 | 59.6 | 16.9 |\n| MMAU Audio Understanding | 81.8 | **81.9** | 76.9 | 77.2 | 63.5 |\n| MMAR Audio Understanding | **79.8** | **79.8** | 78.5 | 77.7 | 67.3 |\n| MMSU Audio Understanding | 82.1 | 83.0 | **83.3** | 80.2 | 59.9 |\n| LongAudioSpan Long Audio Reasoning | Accuracy **82.7** Rubric **71.8** Chain 48.2 | Accuracy 74.4 Rubric 49.8 Chain 45.1 | Accuracy 79.3 Rubric 65.5 Chain **64.6** | -- | -- |\n| MuchoMusic-RUL Music Understanding | **72.6** | 71.6 | 53.7 | 61.7 | 40.1 |\n| HumMusQA Music Understanding | **75.8** | 75.5 | 71.2 | 66.0 | 63.3 |\n| MusTBench Music Understanding | **50.6** | 49.1 | 40.3 | 44.0 | 29.4 |\n| Audio MultiChallenge Audio Interaction | 71.5 | 57.6 | **71.9** | 63.4 | 57.9 |\n| WildSpeech Audio Interaction | 74.3 | 75.7 | **76.4** | 74.5 | 73.4 |\n| VoiceBench Audio Interaction | 91.6 | **92.9** | 92.3 | 84.1 | 79.8 |\n\n1\\. Evaluation harnesses for Agentic Omni Intelligence: WildClawBench-MM and AgenticVBench use Claude Code, UniClawBench uses OpenClaw, and OmniGAIA uses no harness. For WildClawBench-MM, we evaluate only the multimodal tasks in WildClawBench that involve images, video, or audio.\\\n2\\. FLEURS: ASR and S2TT evaluation results both cover the following 60 languages: Chinese (Mandarin), English, Cantonese, Arabic, German, French, Spanish, Portuguese, Indonesian, Italian, Korean, Russian, Thai, Vietnamese, Japanese, Turkish, Hindi, Malay, Dutch, Urdu, Norwegian, Swedish, Danish, Hebrew, Finnish, Polish, Icelandic, Czech, Filipino, Persian, Greek, Afrikaans, Asturian, Belarusian, Bulgarian, Bengali, Bosnian, Catalan, Cebuano, Estonian, Galician, Gujarati, Croatian, Hungarian, Javanese, Kazakh, Kannada, Kyrgyz, Latvian, Macedonian, Malayalam, Marathi, Punjabi, Romanian, Slovak, Slovenian, Swahili, Tajik, Azerbaijani, and Ukrainian.\\\n3\\. Empty cells (--): scores are not yet available or are not applicable.\n\n### Agentic Omni Understanding\n\nKey information in long audio and video recordings is often scattered across different segments, while complex questions require multiple steps of reasoning across sound and images. Agentic Omni Understanding enables the model to start from the question, plan its approach, call tools, and progressively locate and verify evidence. By focusing computation on relevant content, it improves the accuracy and efficiency of long-form audio-visual understanding. To evaluate this capability, we compare Qwen3.8-Omni-Flash and Gemini 3.8 Flash on OmniVideoBench, Video-MME-v2, and LVOmniBench under two settings: **Static**, where the model directly interprets the input, and an agent mode using **[Qwen Code](https://github.com/qwenLM/qwen-code)**. This comparison shows how introducing agent workflows affects each model’s performance.\n\n| | Qwen3.8-Omni-Flash (Static) | Qwen3.8-Omni-Flash (Qwen Code) | Gemini 3.8 Flash (Static) | Gemini 3.8 Flash (Qwen Code) |\n| -------------------------------------- | --------------------------- | ------------------------------ | ------------------------- | ---------------------------- |\n| OmniVideoBench Audio-Visual Reasoning | 63.4 | 67.8 | 65.2 | **70.1** |\n| Video-MME-v2 Audio-Visual Reasoning | 65.0 | 71.3 | 71.0 | **72.7** |\n| LVOmniBench Long Video Reasoning | 63.3 | **73.6** | 70.7 | 70.7 |\n\n### Text\n\n| | Qwen3.8-Omni-Flash | Qwen3.8- Flash | Qwen3.8- 27B | Qwen3.7- Plus | DeepSeek-V4-Flash-0731 | Claude-Opus-4.6 (Max) |\n| --------------------------------------------------------- | ------------------ | -------------- | ------------ | ------------- | ---------------------- | --------------------- |\n| Coding and Agent | | | | | | |\n| DeepSWE 1.1 Long-horizon software engineering | 57.8 | **58.7** | 42.2 | 16.5 | 54.4 | -- |\n| SWE-bench Pro Long-horizon software engineering | **63.3** | 62.5 | 61.7 | 55.8 | 56.0 | 53.4 |\n| SWE-bench Multilingual Multilingual software engineering | 80.5 | **81.0** | 73.8 | 75.8 | -- | 77.5 |\n| NL2Repo-Bench Repo-level code generation | 48.9 | 48.1 | 42.3 | 41.1 | **54.2** | 47.6 |\n| CoWorkBench Long-horizon office work | **75.3** | 73.9 | 70.7 | 65.1 | 45.1 | 68.2 |\n| General Text Capabilities | | | | | | |\n| IFBench Instruction following | **81.5** | 81.3 | 79.5 | 79.1 | 79.2 | 62.5 |\n| GPQA Diamond Scientific reasoning | 91.0 | **91.7** | 89.2 | 90.3 | 90.8 | 91.3 |\n| HLE Multidisciplinary reasoning | 36.5 | 35.9 | 30.8 | 34.7 | 33.8 | **40.0** |\n| LiveCodeBench v6 Competitive coding | **92.6** | 91.9 | 90.3 | 89.6 | 90.6 | 88.8 |\n\n1\\. DeepSWE 1.1: evaluated with the Claude Code and mini-SWE-agent harnesses, temp=1.0, top\\_p=0.95, 256K context window. We report the highest score across the two harnesses; notably, Qwen3.8-Flash-Next performs best on mini-SWE-agent.\\\n2\\. SWE-bench Pro: except for Claude-Opus-4.6 (Max), for which we report the officially published score, all models are evaluated with the Claude Code harness, temp=1.0, top\\_p=0.95, 256K context window. Problematic tasks were corrected and all baseline models were re-evaluated on the refined benchmark.\\\n3\\. SWE-bench Multilingual: evaluated with the mini-SWE-agent harness, temp=1.0, top\\_p=0.95, 256K context window.\\\n4\\. NL2Repo-Bench: evaluated with the Claude Code harness. To prevent reward hacking, we disable Bash commands that attempt to access the specific repository, such as pip download, pip install and git clone.\\\n5\\. CoWorkBench: an in-house cowork benchmark for evaluating long-horizon office and productivity agent tasks across computer science, finance, law, medical and other productivity domains.\\\n6\\. HLE: judged by GPT-4o.\\\n7\\. Empty cells (--): scores are not yet available or are not applicable.\n\n### Vision\n\n| | Qwen3.8-Omni-Flash | Qwen3.8- Flash | Qwen3.8- 27B | Qwen3.7- Plus | Claude-Opus-4.6 (Max) |\n| --------------------------------------- | --------------------------------------- | ------------------------------- | ------------------------------- | ----------------------------------- | ----------------------------- |\n| Agentic Vision Intelligence | | | | | |\n| ClawEval-MM Multimodal tool use | Pass@3 60.4 Average **61.9** | Pass@3 **64.4** Average 60.4 | Pass@3 57.4 Average 56.9 | Pass@3 57.4 Average 60.1 | Pass@3 52.5 Average 54.7 |\n| AndroidWorld Mobile use | **87.1** | 84.5 | 81.9 | 81.0 | 62.0 |\n| Vision2Web Visual web development | 62.9 | **64.0** | 62.9 | 42.1 | -- |\n| General Vision Capabilities | | | | | |\n| ERQA Embodied intelligence | 71.0 | **72.3** | 65.5 | 69.8 | 40.8 |\n| LVBench Long video understanding | **76.9** | 76.6 | 72.4 | 76.2 | 63.0 |\n| RealWorldQA Real-world perception | 87.7 | **88.5** | 85.9 | 86.9 | 73.9 |\n| MathVision Visual math problem solving | Without CI **91.8** With CI **96.2** | Without CI 90.6 With CI 95.7 | Without CI 90.0 With CI 94.6 | Without CI 90.3 With CI 88.4 | Without CI 65.5 With CI -- |\n| CharXiv (RQ) Scientific chart analysis | Without CI 83.5 With CI **91.4** | Without CI 84.6 With CI 90.6 | Without CI 83.7 With CI 90.2 | Without CI **85.8** With CI 85.9 | Without CI 66.0 With CI -- |\n\n1\\. ClawEval-MM: scores are reported as \"pass@3 / average score\". Pass@3 measures the percentage passed in at least one of three trials, and the average score is the mean score across the three trials.\\\n2\\. RecreationBench: an in-house long-horizon application-recreation benchmark for evaluating hybrid-agent abilities spanning five platforms — desktop (Ubuntu, macOS, Windows), mobile (Android) and web.\\\n3\\. OSWorld 2.0: scores are reported as \"binary / partial\". The binary score is the percentage of tasks that receive the full task reward, while the partial score aggregates the partial rewards obtained across all tasks.\\\n4\\. Vision2Web: scores are reported as the average over the frontend, webpage and website categories, using the Claude Code harness and judged by gpt-5.4-2026-03-05.\\\n5\\. MathVision, CharXiv (RQ): scores are reported as \"without CI / with CI\". A small number of incorrect ground-truth annotations in MathVision were corrected after manual verification. Our model's score is evaluated using a fixed prompt, e.g. \"Please reason step by step, and put your final answer within \\\\boxed{}.\" For other models, we report the higher score between runs with and without the \\\\boxed{} formatting.\\\n6\\. Empty cells (--) indicate scores not yet available or not applicable.\n\n### Throughput & Latency\n\nThe following results show the observed throughput and latency of the Qwen3.8-Omni-Flash-Realtime API under different input conditions, reflecting the performance users experience in production environments.\n\n| Input Scenario | Text Output TPS (Tokens/s) | Time to First Token (ms) | Time to First Audio Packet (ms) | Audio Generation RTF |\n| ------------------------ | -------------------------- | ------------------------ | ------------------------------- | -------------------- |\n| Realtime API Performance | | | | |\n| Audio 6s | 84.87 | 591.26 | 978.36 | 0.1538 |\n| Audio 12s | 83.18 | 604.80 | 982.74 | 0.1537 |\n| Audio 20s | 81.06 | 617.98 | 1026.39 | 0.1538 |\n| Audio-Visual 6s | 84.89 | 837.96 | 1214.73 | 0.1524 |\n| Audio-Visual 12s | 84.33 | 911.85 | 1268.07 | 0.1527 |\n| Audio-Visual 20s | 83.00 | 981.01 | 1350.49 | 0.1528 |\n\n### Supported Languages\n\n| Capability | Languages | Chinese Dialects |\n| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Speech Recognition** | **74 languages:** Afrikaans, Arabic, Asturian, Azerbaijani, Basque, Belarusian, Bengali, Bosnian, Bulgarian, Cantonese, Catalan, Cebuano, Chinese, Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Hebrew, Hindi, Hungarian, Icelandic, Indonesian, Interlingua, Italian, Japanese, Javanese, Kannada, Kazakh, Korean, Kyrgyz, Lingala, Latvian, Lithuanian, Macedonian, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Norwegian Bokmål, Norwegian Nynorsk, Oriya, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian, Slovak, Slovenian, Spanish, Swahili, Swedish, Tajiki, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uyghur, and Vietnamese | **39 dialects:** Northeastern Mandarin, Guizhou dialect, Guangdong Cantonese, Henan dialect, Hong Kong Cantonese, Shanghainese, Shaanxi dialect, Tianjin dialect, Taiwanese Mandarin, Yunnan dialect, Anhui dialect, Fujian dialect, Gansu dialect, Guangdong Mandarin, Hubei dialect, Hunan dialect, Jiangxi dialect, Shandong dialect, Shanxi dialect, Sichuanese, Guangxi dialect, Hainan dialect, Chongqing dialect, Changsha dialect, Hangzhou dialect, Hefei dialect, Yinchuan dialect, Zhengzhou dialect, Shenyang dialect, Wenzhou dialect, Wuhan dialect, Kunming dialect, Taiyuan dialect, Nanchang dialect, Jinan dialect, Lanzhou dialect, Nanjing dialect, Hakka, and Southern Min |\n| **Speech Generation** | **29 languages:** Chinese, English, German, Italian, Portuguese, Spanish, Japanese, Korean, French, Russian, Thai, Indonesian, Arabic, Vietnamese, Turkish, Finnish, Polish, Hindi, Dutch, Czech, Urdu, Tagalog, Swedish, Danish, Hebrew, Icelandic, Malay, Norwegian, and Persian | **7 dialects:** Sichuanese, Beijing dialect, Tianjin dialect, Nanjing dialect, Shaanxi dialect, Cantonese, and Southern Min |\n\n## Getting Started with Qwen3.8-Omni-Flash\n\n### API Usage\n\nQwen3.8-Omni-Flash officially supports `reasoning_effort` to adjust reasoning depth and control costs:\n\n- `xhigh` (default): for complex tasks that require in-depth analysis.\n- `medium`: balances accuracy and speed.\n- `low`: efficient reasoning optimized for speed and cost.\n\nIn addition, `preserve_thinking` is enabled by default across all scenarios for the best out-of-the-box experience.\n\nQwen3.8-Omni-Flash supports industry-standard protocols, including OpenAI-compatible Chat Completions and Responses APIs. Examples follow:\n\n```python\n\"\"\"\nEnvironment variables:\n DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n DASHSCOPE_BASE_URL: (optional) Base URL for compatible-mode API.\n - Beijing: https://dashscope.aliyuncs.com/compatible-mode/v1\n - Singapore: https://dashscope-intl.aliyuncs.com/compatible-mode/v1\n\"\"\"\nfrom openai import OpenAI\nimport os\n\napi_key = os.environ.get(\"DASHSCOPE_API_KEY\")\nif not api_key:\n raise ValueError(\n \"DASHSCOPE_API_KEY is required. \"\n \"Set it via: export DASHSCOPE_API_KEY='your-api-key'\"\n )\n\nclient = OpenAI(\n api_key=api_key,\n base_url=os.environ.get(\n \"DASHSCOPE_BASE_URL\",\n \"https://dashscope-intl.aliyuncs.com/compatible-mode/v1\",\n ),\n)\n\nmessages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg\"\n },\n },\n {\n \"type\": \"input_audio\",\n \"input_audio\": {\n \"data\": \"https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav\",\n \"format\": \"wav\"\n },\n },\n {\"type\": \"text\", \"text\": \"Please describe the image and tell me what is being said in the audio.\"},\n ],\n },\n]\n\ncompletion = client.chat.completions.create(\n model=\"qwen3.8-omni-flash\",\n messages=messages,\n extra_body={\n \"enable_thinking\": True,\n # \"preserve_thinking\": True,\n },\n reasoning_effort=\"xhigh\", # supported levels are xhigh, medium, and low\n stream=True,\n)\n\nreasoning_content = \"\"\nanswer_content = \"\"\nis_answering = False\nprint(\"\\n\" + \"=\" * 20 + \"Reasoning\" + \"=\" * 20 + \"\\n\")\n\nfor chunk in completion:\n if not chunk.choices:\n print(\"\\nUsage:\")\n print(chunk.usage)\n continue\n\n delta = chunk.choices[0].delta\n\n if hasattr(delta, \"reasoning_content\") and delta.reasoning_content is not None:\n if not is_answering:\n print(delta.reasoning_content, end=\"\", flush=True)\n reasoning_content += delta.reasoning_content\n\n if hasattr(delta, \"content\") and delta.content:\n if not is_answering:\n print(\"\\n\" + \"=\" * 20 + \"Answer\" + \"=\" * 20 + \"\\n\")\n is_answering = True\n print(delta.content, end=\"\", flush=True)\n answer_content += delta.content\n```\n\n### Best Practices for Controllable Audio-Visual Captioning\n\n**Content organization and level of detail:** Organize video descriptions, OCR extraction, audio descriptions, and speech transcripts into sections covering visuals, speech, music, sound effects, and ambient sounds. Preserve the original text, speaker identities, and corresponding time ranges to make retrieval and verification easier. Specify a target length in the prompt to control detail, for example, `Describe the video in approximately 2000–3000 words.`, and adjust it to the video’s duration and information density.\n\nPrompt Example\n\n````text\nProvide a detailed description of the video.\n\nMake sure your description covers every one of the following dimensions:\n\nVisual\n- Subjects and characters: appearance, clothing, gender/age cues, identity, distinctive features\n- Actions and events in chronological order, and how the scene evolves over time\n- Setting and background: location, environment, time of day\n- Spatial layout and relations between subjects/objects; counts and quantities\n- On-screen text: captions, titles, subtitles, logos, UI — exact content and appearance\n- Visual style: colors, lighting, camera shots, angles, and camera movement\n\nAudio\n- Speech: the exact spoken content, transcribed verbatim\n- Speakers: who is speaking (mapped to the on-screen person or voice-over), with accent, tone, gender/age cues\n- Speaking state: prosody, emotion, volume, and speaking style\n- Music: presence, genre/mood, and lyrics if any\n- Sound effects and ambient/background sounds\n- Non-speech vocalizations: laughter, crying, applause, etc.\n\nAudio-visual correspondence\n- Which speech or sound aligns with which on-screen person or visual event\n- The timing of each event, expressed with timestamps\n\nIt should explicitly include three sections:\n\n1. A structured chronological storyline of **every noticeable audio and visual details**\n2. A structured list of all visible text. For each text element, include start timestamp, end timestamp, the exact text content, the appearance characteristics. If no text appears, explicitly state so.\n3. A structured speech-to-text transcription, include speaker (corresponding to the character or voice-over in Section 1, including their accent and tone), exact spoken content, start timestamp, end timestamp, and speaking state (prosody, emotion, and style). If no speech appears, explicitly state so.\n\nAside from these three required sections, you are free to organize any additional content in any way you find helpful. This additional content can include global information about the entire video or localized information about specific moments. You may choose the topic of this extra content freely.\n\nRules:\n\n- Add as much descriptive detail as possible.\n- Do not use Markdown bold formatting.\n- Carefully look at frames and listen to the audio, making sure no detail is overlooked.\n\nOutput Format:\n```\n## Storyline\n\n - \n\n\n - \n\n\n - \n\n\n...\n\n## Visible Text\n\n - \n“”: \n“”: \n\n - \n“”: \n“”: \n“”: \n\n - \n“”: \n\n...\n\n## Speakers and Transcript\n\nSpeaker profiles:\n - \n - \n - \n...\n\n - \nSpeaker: \nState: \nContent: “”\n\n - \nSpeaker: \nState: \nContent: “”\n\n - \nSpeaker: \nState: \nContent: “”\n\n...\n\n## \n\n\n\n## \n\n\n\n...\n```\n````\n\n**Structured output and schema constraints:** Include task instructions and a complete JSON Schema in the prompt, specifying field meanings, types, required fields, and constraints to support programmatic parsing and downstream use.\n\nPrompt Example\n\n```text\nDescribe the audio and visual content in detail in English, organized into scenes and events, following the JSON Schema below.\nAll timestamps must be relative to the beginning of the video. End times must not precede start times or exceed the video duration. Each event must fall within the time range of its parent scene.\nInclude only information directly supported by the audio or video. Do not guess or invent details. Do not infer causality merely because a sound and an action occur at the same time.\nReturn only valid JSON, without Markdown fences or commentary.\n\nJSON Schema:\n{\n \"$defs\": {\n \"Event\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"time_range\": {\n \"$ref\": \"#/$defs/TimeRange\",\n \"description\": \"Time range of the event\"\n },\n \"participants\": {\n \"description\": \"People, animals, or objects involved, named by observable features; use consistent names for the same participant\",\n \"items\": {\n \"type\": \"string\"\n },\n \"title\": \"Participants\",\n \"type\": \"array\"\n },\n \"action\": {\n \"description\": \"Specific actions, interactions, and observable outcomes\",\n \"title\": \"Action\",\n \"type\": \"string\"\n },\n \"sounds\": {\n \"description\": \"Sounds heard during the event; use an empty list if none are discernible\",\n \"items\": {\n \"type\": \"string\"\n },\n \"title\": \"Sounds\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"time_range\",\n \"participants\",\n \"action\",\n \"sounds\"\n ],\n \"title\": \"Event\",\n \"type\": \"object\"\n },\n \"Scene\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"time_range\": {\n \"$ref\": \"#/$defs/TimeRange\",\n \"description\": \"Time range of the scene\"\n },\n \"setting\": {\n \"description\": \"Environment, spatial layout, and main visual features\",\n \"title\": \"Setting\",\n \"type\": \"string\"\n },\n \"events\": {\n \"description\": \"Events in chronological order; use an empty list if there are none\",\n \"items\": {\n \"$ref\": \"#/$defs/Event\"\n },\n \"title\": \"Events\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"time_range\",\n \"setting\",\n \"events\"\n ],\n \"title\": \"Scene\",\n \"type\": \"object\"\n },\n \"TimeRange\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"start_seconds\": {\n \"description\": \"Start time in seconds relative to the beginning of the video\",\n \"minimum\": 0,\n \"title\": \"Start Seconds\",\n \"type\": \"number\"\n },\n \"end_seconds\": {\n \"description\": \"End time in seconds; must not precede the start time\",\n \"minimum\": 0,\n \"title\": \"End Seconds\",\n \"type\": \"number\"\n }\n },\n \"required\": [\n \"start_seconds\",\n \"end_seconds\"\n ],\n \"title\": \"TimeRange\",\n \"type\": \"object\"\n }\n },\n \"additionalProperties\": false,\n \"properties\": {\n \"summary\": {\n \"description\": \"An overview of the main content of the video\",\n \"title\": \"Summary\",\n \"type\": \"string\"\n },\n \"scenes\": {\n \"description\": \"Scenes in chronological order; group continuous footage with a consistent setting into one scene\",\n \"items\": {\n \"$ref\": \"#/$defs/Scene\"\n },\n \"title\": \"Scenes\",\n \"type\": \"array\"\n }\n },\n \"required\": [\n \"summary\",\n \"scenes\"\n ],\n \"title\": \"CaptionResult\",\n \"type\": \"object\"\n}\n```\n\n### Installing Qwen-MM-Plugins in an Agent Harness\n\n[Qwen-MM-Plugins](https://github.com/QwenLM/Qwen-MM-Plugins) is a multimodal plugin suite for agent harnesses. It gives agents the ability to understand images, audio, video, and documents, as well as maintain memory for long-form video and create content. It supports agent harnesses including Codex, Claude Code, Qwen Code, Gemini CLI, Qoder, CodeBuddy, and OpenClaw. We also welcome contributions from community developers.\n\nYou can enter the following request directly in your usual office agent:\n\n```text\nHelp me install the core, api, and omni-related plugins from https://github.com/QwenLM/Qwen-MM-Plugins.\n```\n\n![Installing Qwen-MM-Plugins in an office agent using natural language](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen3.8-Omni-Flash/mm-plugins-install-20260917.en.png)\n\nAlternatively, install from the command line:\n\n```bash\n# Run the official guided installer:\ncurl -fsSL https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh | bash\n```\n\nIn the menu, select:\n\n- Install\n- The agent harness you use\n- The Omni plugins you need\n\n| Plugin | Capability |\n| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |\n| `core` | Read images, video frames, PDFs, Office documents, code, data, and 3D files |\n| `api` | Image understanding, OCR, object localization, audio-visual transcription, speaker diarization, event analysis, and image segmentation |\n| `omni-chatcut` | Create MVs, long-form film commentary, and video speech translations |\n| `omni-video2note` | Turn video tutorials into PDF notes with key screenshots |\n| `omni-skill-creator` | Turn instructional videos, screen recordings, or operation demonstrations into reusable Agent Skill.md files |\n| `omni-memory` | Build memory for people, dialogue, sounds, and events in long-form videos |\n\nAfter installation, restart the agent harness or create a new task.\n\n```text\n@song.mp3 Generate a complete MV based on the song's rhythm and content.\n@short_drama.mp4 Translate the video into English while preserving the original speakers' voice characteristics where possible.\n@movie.mp4 Create a film commentary video with Chinese narration and subtitles.\n\n@weekly_report_sop.mp4 Turn this screen recording of writing a weekly report into a Skill.md file.\n@tutorial.mp4 Turn the tutorial into PDF notes with key screenshots, timestamps, and step-by-step instructions.\n@documentary.mp4 Build audio-visual memory that records people, dialogue, sounds, and important events.\n```\n\n## Getting Started with Qwen3.8-Omni-Flash-Realtime\n\n### API Usage\n\nQwen3.8-Omni-Flash-Realtime supports connections over WebSocket and WebRTC. Running the basic examples below opens the camera and microphone for a real-time audio-visual conversation. We recommend using headphones.\n\n```python\n# Run pip install websocket-client pyaudio dashscope opencv-python -U to install dependencies\n\"\"\"\nEnvironment variables:\n DASHSCOPE_API_KEY: Your API Key from https://platform.qianwenai.com/home\n DASHSCOPE_BASE_URL: (optional) Base URL for realtime API.\n - Beijing: wss://dashscope.aliyuncs.com/api-ws/v1/realtime\n - Singapore: wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime\n\"\"\"\nimport os\nimport base64\nimport time\nimport pyaudio\nimport cv2\nfrom dashscope.audio.qwen_omni import MultiModality, AudioFormat,OmniRealtimeCallback,OmniRealtimeConversation\nimport dashscope\n\nurl = f'wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime'\ndashscope.api_key = os.getenv('DASHSCOPE_API_KEY')\n# Determine the voice\nvoice = 'Tina'\n# Determine the model\nmodel = 'qwen3.8-omni-flash-realtime'\n# Determine the model role\ninstructions = \"You are Qwen-Omni, a helpful assistant.\"\nvideo_fps = 1\nvideo_size = (1280, 720)\nclass SimpleCallback(OmniRealtimeCallback):\n def __init__(self, pya):\n self.pya = pya\n self.out = None\n def on_open(self):\n # Initialize audio output stream\n self.out = self.pya.open(\n format=pyaudio.paInt16,\n channels=1,\n rate=24000,\n output=True\n )\n def on_event(self, response):\n if response['type'] == 'response.audio.delta':\n # Play audio\n self.out.write(base64.b64decode(response['delta']))\n elif response['type'] == 'conversation.item.input_audio_transcription.delta':\n # Streaming preview: text is the confirmed prefix, stash is the confirmed suffix\n preview = response.get('text', '') + response.get('stash', '')\n print(f\"\\r[User] {preview}\", end='', flush=True)\n elif response['type'] == 'conversation.item.input_audio_transcription.completed':\n # Transcription completed, print the final text and a new line\n print(f\"\\r[User] {response['transcript']}\")\n elif response['type'] == 'response.audio_transcript.done':\n # Print the assistant's response text\n print(f\"[LLM] {response['transcript']}\")\n\n# 1. Initialize audio device\npya = pyaudio.PyAudio()\n# 2. Create callback function and session\ncallback = SimpleCallback(pya)\nconv = OmniRealtimeConversation(model=model, callback=callback, url=url)\n# 3. Establish connection and configure session\nconv.connect()\nconv.update_session(output_modalities=[MultiModality.AUDIO, MultiModality.TEXT], voice=voice, instructions=instructions)\n# 4. Initialize audio input stream\nmic = pya.open(format=pyaudio.paInt16, channels=1, rate=16000, input=True)\ncamera = cv2.VideoCapture(0)\nnext_frame_at = 0.0\n# 5. Main loop to process audio and video input\ntry:\n if not camera.isOpened():\n raise RuntimeError(\"Cannot open camera 0.\")\n camera.set(cv2.CAP_PROP_FRAME_WIDTH, video_size[0])\n camera.set(cv2.CAP_PROP_FRAME_HEIGHT, video_size[1])\n camera.set(cv2.CAP_PROP_BUFFERSIZE, 1)\n print(f\"Conversation started with camera ({video_fps} fps, {video_size[0]}x{video_size[1]}), speak into the microphone (Ctrl+C to exit)...\")\n while True:\n audio_data = mic.read(3200, exception_on_overflow=False)\n conv.append_audio(base64.b64encode(audio_data).decode())\n success, frame = camera.read()\n if not success:\n raise RuntimeError(\"Cannot read a camera frame.\")\n if time.monotonic() >= next_frame_at:\n frame = cv2.resize(frame, video_size)\n success, image = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 80])\n if not success:\n raise RuntimeError(\"Cannot encode a camera frame.\")\n conv.append_video(base64.b64encode(image).decode())\n next_frame_at = time.monotonic() + 1 / video_fps\n time.sleep(0.01)\nexcept KeyboardInterrupt:\n pass\nfinally:\n # Clean up resources\n camera.release()\n conv.close()\n mic.close()\n if callback.out:\n callback.out.close()\n pya.terminate()\n print(\"\\nConversation ended\")\n```\n\n### Qwen-Live Harness\n\n[Qwen-Live Harness](https://github.com/QwenLM/Qwen-Live-Harness) is a comprehensive open-source harness designed around the Qwen3.8-Omni-Flash-Realtime API. It can be installed with a single command and integrated into mainstream agent workflows. It supports task delegation, proactive interaction, long-term memory, and context management, and welcomes contributions from the community.\n\n![Qwen-Live Harness interaction framework](https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/Qwen3.8-Omni-Flash/fig3.png?v=20260916-2053#center)\n\nFigure 2. Qwen-Live Harness Interaction Framework.\n\nInstall and get started:\n\n```bash\nnpm install -g qwen-live-harness\nqwen-live-harness init\nqwen-live-harness\n```\n\n## Citation\n\n```bibtex\n@misc{qwen38omniflash,\n title = {Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.},\n url = {https://qwen.ai/blog?id=qwen3.8-omni-flash},\n author = {{Qwen Team}},\n month = {September},\n year = {2026}\n}\n```\n", "dateCreated": "2026-10-19T01:26:53.998049939+00:00", "date_created": "2026-09-18T07:00:00+00:00", "date_modified": null, "date_published": "2026-09-18T07:00:00+00:00", "description": "Introduction Today, we are launching Qwen3.8-Omni-Flash, our next-generation native omnimodal model. Its core objective is to strengthen agent capabilities in real-world productivity scenarios, advancing omnimodal models from…", "external_url": "https://qwen.ai/blog?id=qwen3.8-omni-flash", "git": { "commit": "ebb0941c5c06a2ab2377342d607535262a3e969e", "path": "items/qwen-ai/2026/09/2026-09-18-qwen3-8-omni-flash-omni-senses-agentic-delivery.md", "permalink": "https://github.com/aymericbeaumet/aggr-instance/blob/ebb0941c5c06a2ab2377342d607535262a3e969e/items/qwen-ai/2026/09/2026-09-18-qwen3-8-omni-flash-omni-senses-agentic-delivery.md" }, "id": "urn:aggr:item:68d832950a358a8a4fd8195eae6db1c7f1f65974", "isBasedOn": { "@id": "https://qwen.ai/blog?id=qwen3.8-omni-flash" }, "mainEntity": { "@id": "https://qwen.ai/blog?id=qwen3.8-omni-flash", "@type": "CreativeWork", "archivedAt": { "@id": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-09-18-qwen3-8-omni-flash-omni-senses-agentic-delivery/#webpage" }, "author": [ { "@type": "Person", "name": "QwenTeam" } ], "datePublished": "2026-09-18T07:00:00+00:00", "headline": "Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.", "image": { "@type": "ImageObject", "caption": null, "height": 90, "url": "https://img.alicdn.com/imgextra/i1/O1CN01OvSn2RXYZRD29R4a_!!6000000000892-2-tps-1060-636.png", "width": 160 }, "timeRequired": "PT28M", "url": "https://qwen.ai/blog?id=qwen3.8-omni-flash", "wordCount": 6284 }, "name": "Archived snapshot: Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.", "preview": { "alt": null, "color": null, "height": 90, "placeholder": { "data_url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=", "hash": "" }, "url": "https://img.alicdn.com/imgextra/i1/O1CN01OvSn2RXYZRD29R4a_!!6000000000892-2-tps-1060-636.png", "width": 160 }, "reading_minutes": 28, "replicated_at": null, "source": "qwen-ai", "source_name": "Qwen", "summary": null, "tags": [ "release" ], "temporalCoverage": "2026-10-19T01:26:53.998049939+00:00", "title": "Qwen3.8-Omni-Flash: Omni Senses. Agentic Delivery.", "url": "https://aggr.aymericbeaumet.com/items/qwen-ai/2026-09-18-qwen3-8-omni-flash-omni-senses-agentic-delivery/", "word_count": 6284 }