Kimi Research Blog | Moonshot AI

PerceptionBench

2026-07-16T00:00:00+00:00

Introducing PerceptionBench

Evaluating Atomic Visual Perception in Multimodal Large Language Models

Authors Kimi Team

Overview

We are releasing PerceptionBench, a benchmark that isolates visual perception and evaluates it as a set of atomic capabilities—discovered from how today's models fail, not defined in advance. From frontier-model failures across 42 benchmarks, we derive 10 atomic perceptual capabilities and construct 3,000 verified questions, each isolating a single capability and answerable by looking, with no reasoning or external knowledge required.

Across sixteen frontier MLLMs, no model reaches 60% accuracy, and perception-related hallucination is the weakest capability on average. Models with nearly identical overall scores can diverge sharply in what they actually perceive. PerceptionBench is built to expose exactly where perception breaks, and to drive progress toward multimodal AI that sees faithfully and consistently.

The Dataset

Guided by the induced taxonomy, we select the most informative failures from the source benchmarks, decompose them into atomic sub-questions, and author additional questions on supplemented images. The retained and constructed samples together form an in-house pool of 17,000+ verified questions. The released benchmark subsamples 3,000 verified questions from the constructed portion—60% decomposed from attributed model failures and 40% newly authored—with category-level balancing and difficulty stratification to isolate atomic perceptual capabilities from confounding factors. The released benchmark distinguishes itself through three core design principles:

Failure-Driven Taxonomy: Every category is discovered from real model failures, attributed to the earliest erroneous step across 42 existing benchmarks.
Ten Atomic Perceptual Categories: Visual Relation, Counting, Attribute, Depth & 3D, Localization, Comparison, Fine-grained Recognition, Context Integration, OCR, and Hallucination.
Perception, Not Reasoning or Knowledge: Samples are curated, decomposed, and difficulty-balanced so that difficulty stems from perception rather than reasoning or external knowledge.

Each source benchmark captures a narrow slice of perception errors, and these slices overlap only weakly (mean pairwise weighted Jaccard 0.20). No single benchmark—or small group of them—covers perception as a whole, which motivates a capability-centric benchmark that aggregates and rebalances these fragmented views.

Distribution of Tasks per Category

Statistics	Number
Data
Total	3,000
Atomic perceptual categories	10
Task Categories
Depth 3D Perception Error	330 (11.00%)
Visual Counting Error	330 (11.00%)
Fine-Grained Recognition Error	290 (9.67%)
Visual Relation Error	330 (11.00%)
Visual Attribute Error	330 (11.00%)
Visual Localization Error	330 (11.00%)
Visual Comparison Error	279 (9.30%)
Context Integration Error	255 (8.50%)
Hallucination	271 (9.03%)
OCR Error	255 (8.50%)

Using PerceptionBench to Compare Models

Conclusion

PerceptionBench is a simple but challenging benchmark for evaluating the atomic visual perception of frontier models. It provides a capability-level standard for measuring and diagnosing the visual perception boundaries of multimodal models. We are open-sourcing the PerceptionBench dataset and evaluation code to help the community address the visual perception gap.

Download the Data: https://github.com/MoonshotAI/PerceptionBench

Kimi K3

2026-07-16T00:00:00+00:00

Today, we are introducing Kimi K3 — our most capable model. Kimi K3 is a 2.8T-parameter model built on our Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

While its overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 demonstrated frontier-level performance across our evaluation suite, consistently outperforming other tested models.

Kimi K3 is available today on Kimi.com, Kimi Work, Kimi Code, and the Kimi API. At launch, Kimi K3 will use max thinking effort by default, with low- and high-effort modes to be introduced in subsequent updates. We are currently working closely with inference partners and open-source maintainers to align technical details and ensure a reliable rollout across the ecosystem. The full model weights will be released by July 27, 2026. Further details on the architecture, training, and evaluations will be released alongside the Kimi K3 technical report.

An Open 3T-Class Model

Kimi K3 is the first open model to reach 2.8 trillion parameters. It marks the latest step in Kimi's sustained push at the scaling frontier: for nine of the past twelve months, Kimi models have set the upper bound of open-model sizes.

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth. We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework. Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to Kimi K2, allowing the model to convert compute into intelligence more effectively.

Coding

Kimi K3 has strong long-horizon coding performance. Operating with minimal human oversight, it can sustain long engineering sessions, navigate massive repositories, and orchestrate terminal tools.

Kimi K3 also excels in tasks blending software engineering with visual reasoning — it leverages screenshots and visuals to optimize game dev, frontend, and CAD.

The case studies below show how Kimi K3's coding capability translates into open-ended software creation and scientific research.

Kernel Optimization

We tested the models' capability to optimize GPU kernels. Each model works independently in an identical sandbox, with up to 24 hours to profile, rewrite, and benchmark four tasks spanning AttnRes, KDA, and a 512-head-dimension MLA kernel across NVIDIA Hopper GPUs and GPGPU from an alternative vendor. Kimi K3 performed competitively with Fable 5 (with fallback) and substantially outperformed Opus 4.8, GPT 5.6 Sol, and GPT 5.5.

Claude Fable 5 was evaluated by a third party, and its results may include fallback behavior. Across most models, some trajectories include small, acceptable precision shortcuts that remain within our numerical tolerance. GPGPU denotes general-purpose GPUs used for computation beyond graphics rendering.

In the late stages of Kimi K3 development, an early version of Kimi K3 handled the majority of the team's kernel optimization works.

GPU Compiler Development

We further tested whether Kimi K3 could build a GPU programming system from scratch. Kimi K3 developed MiniTriton, a compact Triton-like compiler with its own tile-level IR layer over MLIR, optimization passes, and a PTX code-generation pipeline. Across supported roofline benchmarks, MiniTriton delivers performance on par with or better than Triton and torch.compile — beating Triton on certain workloads. Beyond microbenchmarks, MiniTriton sustains end-to-end nanoGPT training with stable convergence, the loss curve closely tracking the reference with only minor divergence — validating the full pipeline on a realistic workload. These results demonstrate that Kimi K3 can build a coherent end-to-end compiler — from DSL frontend and IR passes to PTX codegen and runtime — rather than isolated kernels; its from-scratch Tensor Core path already rivals Triton’s extensively optimized stack.

Game Dev and Digital Creation

Kimi K3 combines strong 3D reasoning, coding, and vision capabilities to turn concepts, images, and videos into fully playable interactive experiences. Kimi K3 achieves true "vision in the loop" by seamlessly iterating between code and live screenshots—instantly seeing and refining outputs.

Chip Design

As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimized, and verified the chip using open-source EDA tools on the Nangate 45nm library. Within 4 mm², the chip closes timing at 100 MHz and sustains over 8,700 tokens/s decode throughput in simulation, packing 1.46M standard cells, 0.277 MB of SRAM, and an INT4 MAC array with fused dequantization. A chip built by a model, for a model, reflects K3's long-horizon agentic capabilities.

Coding for Research

Kimi K3 bridges scientific literature and executable code, autonomously implementing, validating, and analyzing complex computational research workflows.

In one case, Kimi K3 completed in about two hours what would typically require one to two weeks of work by an experienced researcher. To reproduce the I–Love–Q universal relations in computational astrophysics, it reviewed and cross-validated 20+ papers, implemented the full numerical pipeline, evaluated 300+ equations of state, identified inconsistencies in published formulas, generated 3,000+ lines of Python code, and produced an interactive HTML dashboard for exploring the results.

Knowledge Work

Kimi K3 advances end-to-end knowledge work. Beyond public benchmarks, Kimi K3 (max) demonstrates consistent gains across our internal evaluations, which are derived from recurring patterns and challenges observed in real-world user-agent workflows. These consistent advantages across distinct production-oriented workflows reflect a broad improvement in Kimi K3's agentic knowledge work capabilities.

Research with Interactive Visualization

Below are a few examples of what Kimi K3 in Kimi Work can produce across financial consulting and scientific research:

Case 1: Interactive 42 years of AI ASIC industry research website

An interactive research report you can drill into: 42 years of the ASIC industry, created through 120+ rounds of recursive self-improvement. Kimi K3 transforms evidence into bespoke charts, animated diagrams, and interactive visual narratives. It pulled data via 2.8k+ web searches/fetches and 1.1k+ terminal data pulls, across 11k+ pages spanning 87 quarterly reports and 99 original PDFs.

Case 2: Fusion Industry Research

A consulting-style industry report with interactive visualizations—including timelines, Funnel Chart, Range Bar Chart, Gantt Charts, and publication-quality slides.

Case 3: GWTC-5 Gravitational-wave Analysis

An analysis of 391 gravitational-wave events using 20+ concurrent subagents, producing 7 scientific visualizations, 2 tables, and a literature synthesis from 10+ papers.

Kimi K3 is also particularly effective at producing infographic-style presentations, such as the fully editable heatmap and annual report shown below:

Widgets and Dashboard

In Kimi Work, we introduce two new features - Widgets and Dashboard - which make interactions with Kimi K3 more visual and persistent. Widgets let you generate interactive components directly within a chat, with connections to local data or external plugins for continuous updates. Dashboard brings the widgets you care about most into one persistent, personalized view organized around a topic, project, or goal.

Video Editing

Kimi K3 excels at motion design, animation, and video editing because its native multimodal architecture understands text, images, and video within the same model.

In one example, K3 created a 3Blue1Brown-style motion-graphics explainer of its own architecture, translating technical ideas into animated diagrams and transitions.

In another, Kimi K3 edited its own teaser video from 56 source clips, handling clip selection, motion-matched cuts, frame-accurate beat synchronization, audio processing, and multiple rounds of revision. A high-density short video like this would typically take an experienced editor one to two working days, or a beginner three to five.

Architecture and Infrastructure

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). KDA provides an efficient foundation for scaling attention, while AttnRes selectively retrieves representations across depth rather than accumulating them uniformly. Together, they form the architectural backbone of a model designed to scale well beyond the trillion-parameter regime.

Kimi K3 uses Stable LatentMoE, effectively activating 16 of 896 experts. At this level of sparsity, routing and optimization become first-order challenges. Quantile Balancing derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter, while Per-Head Muon extends Muon by optimizing attention heads independently for more adaptive learning at scale. Sigmoid Tanh Unit (SiTU) and Gated MLA improve activation control and attention selectivity respectively. Together, these advances enable stable and efficient training at the 2.8-trillion-parameter scale.

Kimi K3 applies quantization-aware training from the SFT stage onward, using MXFP4 weights with MXFP8 activations for broad hardware compatibility. To prevent expert imbalance from degrading throughput at large expert-parallel scales, we introduce a fully balanced expert-parallel training method with static shapes and no host synchronization on the critical path. Since inference efficiency likewise benefits from larger high-bandwidth communication domains, we recommend deploying Kimi K3 on supernode configurations with 64 or more accelerators. Finally, as KDA poses new challenges for conventional prefix caching, we have contributed a corresponding implementation to the vLLM community, to be released alongside the model. KDA with prefill cache allows us to serve Kimi K3 at a highly competitive token price despite its scale and long context.

More technical details will be available in our coming report.

Availability

Kimi K3 Agents: Download or update to the latest Kimi app from your mobile app store, available on iOS, Android, and HarmonyOS, or visit kimi.com.
Work with Kimi K3: Download the latest Kimi Work desktop app, version 3.1.0 or later, available for Windows and Apple silicon Macs.
Code with Kimi K3: Run Kimi Code in your terminal and select Kimi K3 using the /model command.
Build with the Kimi API: Visit the Kimi API Platform and select kimi-k3. Pricing is $0.30/MTok for cache-hit input, $3.00/MTok for cache-miss input, and $15.00/MTok for output. Powered by Mooncake's disaggregated inference architecture, the official Kimi API achieves a cache hit rate above 90% in coding workloads.
Bring Kimi to your organization: Kimi Enterprise provides enterprise-grade data privacy and member management, with complete separation between personal and organization accounts. Visit the pricing page and select “Get Kimi Enterprise” to subscribe for your team.

Full Benchmark Table

Footnotes

All Kimi K3 results reported below are obtained with the reasoning effort set to 'max', setting temperature = 1.0 and top-p = 1.0. Depending on the benchmark, each model is evaluated under one of three agentic harnesses — Kimi Code, Claude Code, or Codex — as specified in the notes below.

Coding benchmarks

DeepSWE. Kimi K3 is evaluated with the Kimi Code harness. The GLM-5.2 score is taken from the GLM-5.2 release blog (https://z.ai/blog/glm-5.2); all remaining scores are from the official DeepSWE leaderboard (https://deepswe.datacurve.ai/), under which Kimi K3 attains 67.3 with the mini-SWE-agent harness. We report the DeepSWE v1.1 tasks.
Terminal-Bench 2.1. Kimi K3 is evaluated with the Kimi Code harness. For all other models, we report the best score across harnesses: GLM-5.2 with Claude Code (https://z.ai/blog/glm-5.2); Claude Opus 4.8 and Claude Fable 5 with Terminus 2 (https://artificialanalysis.ai/evaluations/terminalbench-v2-1); GPT 5.5 and GPT 5.6 Sol with Codex (https://openai.com/index/previewing-gpt-5-6-sol/).
Program Bench. Kimi K3 is evaluated with the Kimi Code harness. The GLM-5.2 score is from https://z.ai/blog/glm-5.2; all other scores are from https://www.vals.ai/benchmarks/programbench.
SWE Marathon. Kimi K3, Claude Opus 4.8, and Claude Fable 5 are evaluated with the Claude Code harness; GPT-5.6 Sol is evaluated with the Codex harness. The GLM-5.2 score is from https://z.ai/blog/glm-5.2. Our evaluation is based on an H20-calibrated branch of the official v1.1 tasks (https://www.swe-marathon.org/): the Docker images, performance gates, and reference oracles for the GPU tasks have been recalibrated for H20, while the correctness and anti-cheat validators remain unchanged. Additionally, Claude Fable 5 hit fallbacks on 35% of the tasks in our evaluation, which may have negatively impacted its measured performance.
FrontierSWE. Kimi K3 is evaluated with the Kimi Code harness and GPT-5.6 Sol with the Codex harness; all other results are from https://www.frontierswe.com/. Dominance scores are recomputed from the raw scores using the official evaluation script and are current as of July 16, 2026.
PostTrain Bench. Scores for GLM-5.2, GPT-5.5, and Claude Opus 4.8 are adopted from the official PostTrainBench (https://posttrainbench.com/) results. Kimi K3, Claude Fable 5, and GPT-5.6 Sol are evaluated with the official Harbor implementation at maximum reasoning effort, averaged over three runs on H20 GPU (instead of H100 in the official setting) — Kimi K3 and Claude Fable 5 with the Claude Code harness, and GPT-5.6 Sol with the Codex harness.
MLS Bench Lite. Kimi K3 is evaluated with the Kimi Code harness; GLM-5.2 and the Claude models with the Claude Code harness; GPT-5.5 and GPT-5.6 Sol with the Codex harness.
KCB 2.0. Kimi K3 is evaluated with both the Kimi Code and Claude Code harnesses; GLM-5.2, Claude Opus 4.8, and Claude Fable 5 with the Claude Code harness; GPT-5.5 and GPT-5.6 Sol with the Codex harness. All models are evaluated at maximum reasoning effort, except GPT-5.5, which uses the "xhigh" setting. We also note that on this in-house benchmark, 10% of the tasks entered GPT-5.6 Sol's cyber guard.

Productivity and agentic benchmarks

For OfficeQA Pro, each test case provides the agent with the entire PDF corpus, with all PDFs rendered as images and no machine-readable text available.
OfficeQA Pro and SpreadsheetBench 2. Kimi K3, GLM-5.2, Claude Opus 4.8, and Claude Fable 5 are evaluated with the Claude Code harness; GPT 5.5 and GPT 5.6 Sol are evaluated with the Codex harness.
MCP Atlas. All models are evaluated on the 500-task public subset with a 100-turn limit, using Gemini 3.1 Pro as the judge.
AutomationBench. All models are evaluated on the 600-task public subset, following the official GitHub setup in all other respects.
BrowseComp. We adopt the context-compaction strategy used in the Claude model cards, triggered at 300K tokens. When evaluated with a 1M-token context window and no context management, Kimi K3 achieves a score of 90.4. The results of Claude Fable 5, Claude Opus 4.8, GPT 5.6 Sol, and GPT 5.5 are cited from https://www.anthropic.com/news/claude-fable-5-mythos-5 and https://openai.com/index/gpt-5-6/.
GDPval-AA, AA-Briefcase, and APEX-Agents scores are cited from https://artificialanalysis.ai/.

Multimodal benchmarks

Except for ZeroBench, which follows the official setting and is run five times, all multimodal scores are averaged over three runs. MMMU-Pro is evaluated following the official protocol, preserving the original input order and prepending images to the text input.
PerceptionBench. PerceptionBench (https://www.kimi.com/blog/perception-bench) is a benchmark that focuses on atomic visual perception capabilities.

Limitations

Sensitivity to thinking history. K3 was trained in the preserved thinking history mode. If the agent harness fails to pass back all the historical thinking content as required, or if an ongoing session with another model is switched over to K3, generation quality may become highly unstable. We recommend using a harness with verified compatibility, such as Kimi Code, and avoiding switching to K3 in the middle of a session.
Excessive proactiveness. K3's training places particular emphasis on long-horizon, challenging tasks. As a result, when it encounters minor issues or ambiguous user intent during task execution, it may make unexpected decisions on the user's behalf. If your application requires the agent to operate within well-defined boundaries and refrain from excessive improvisation, please impose more explicit behavioral constraints on K3 in the system prompt or in AGENTS.md.
Despite being a highly competitive model overall, K3 nonetheless exhibits a noticeable gap in user experience compared with Claude Fable 5 and GPT 5.6 Sol.

Kimi K2.6

2026-04-20T00:00:00+00:00

We are open sourcing our latest model, Kimi K2.6, featuring state-of-the-art coding, long-horizon execution, and agent swarm capabilities. Kimi K2.6 is now available via Kimi.com, the Kimi App, the API, and Kimi Code.

Long-Horizon Coding

Kimi K2.6 shows strong improvements in long-horizon coding tasks, with reliable generalization across programming languages (e.g., Rust, Go, and Python) and tasks (e.g., front-end, devops, and performance optimization). On Kimi Code Bench, our internal coding benchmark covering diverse complicated end-to-end tasks, Kimi K2.6 demonstrates significant improvements over Kimi K2.5.

Kimi K2.6 demonstrates strong long-horizon coding in complex engineering tasks:

Kimi K2.6 successfully downloaded and deployed the Qwen3.5-0.8B model locally on a Mac. By implementing and optimizing model inference in Zig—a highly niche programming language—it demonstrated exceptional out-of-distribution generalization. Across 4,000+ tool calls, over 12 hours of continuous execution, and 14 iterations, Kimi K2.6 dramatically improved throughput from ~15 to ~193 tokens/sec, ultimately achieving speeds ~20% faster than LM Studio.

Kimi K2.6 autonomously overhauled exchange-core, an 8-year-old open-source financial matching engine. Over a 13-hour execution, the model iterated through 12 optimization strategies, initiating over 1,000 tool calls to precisely modify more than 4,000 lines of code. Acting as an expert systems architect, Kimi K2.6 analyzed CPU and allocation flame graphs to pinpoint hidden bottlenecks and boldly reconfigured the core thread topology (from 4ME+2RE to 2ME+1RE). Despite the engine already operating near its performance limits, Kimi K2.6 extracted a 185% medium throughput leap (from 0.43 to 1.24 MT/s) and a 133% performance throughput gain (soaring from 1.23 to 2.86 MT/s).

In beta tests, K2.6 performs well on long-horizon coding tasks in enterprise evaluations (randomly ordered):

In a no-code environment, AI has to handle every edge case. There's no developer to step in when something doesn't work as expected. K2.6 is noticeably more effective than K2.5 at navigating nuanced API behaviors and recovering when things break, and it runs longer-horizon tasks before hitting a wall. We've seen a real improvement in getting users from idea to deployment compared to K2.5.

What impressed us most about K2.6 is its surgical precision in large codebases. When an initial path is blocked, it is strong at pivoting intelligently: following existing architectural patterns, finding hidden related changes, and keeping fixes scoped to the real problem. That kind of focused adaptability helps Augment Code reduce wasted cycles and deliver faster, more cost-effective agentic coding for enterprise-scale engineering work.

Kimi K2.6's evolution is impressive. It excels on coding tasks at a level comparable to leading closed source models, and offers strong tool calling quality due to its deep understanding of third party frameworks. Kimi K2.6's excellent reliability makes it a great choice for complex and long-horizon engineering tasks.

Kimi K2.6 sets a new level for open-sourced models, especially in long-horizon, agent-style coding workflows. It handles complex, multi-step tasks with stronger instruction following and consistently high code quality. We've seen it sustain extended coding sessions with remarkable stability, far beyond typical models. It also surfaces deep, non-obvious bugs that would normally take significant developer time to uncover. Overall, K2.6 sets a new bar for reliable coding.

Kimi K2.6 demonstrates significant improvements over K2.5 in internal evaluations conducted by CodeBuddy: code generation accuracy increased by 12%, long-context stability improved by 18%, and tool invocation success rate reached 96.60%. Its stronger reasoning capabilities and more consistent output quality provide robust support for ensuring a reliable user experience in CodeBuddy WorkBuddy.

K2.6 is a clear improvement on K2.5 on both our benchmarks (+15%) and in side-by-side comparisons. It seems to have better instruction following, more thorough exploration and reasoning, and less likely to make coding errors or use hacks.

We are thrilled to see another leap in open source models with Kimi K2.6 release, which marks a significant advancement for high-stakes, agentic workflows. The most impactful improvements lie in its long-horizon reliability and instruction following. K2.6 excels at maintaining architectural integrity over extended coding sessions, making it a stable foundation for autonomous agent pipelines, like all the "claws". It demonstrates a measurable leap over K2.5 in long-context tasks, achieving state-of-the-art performance in complex reasoning.

Got an early look at K2.6 and ran it through Hermes Agent. Tool calling and agentic loops feel noticeably tighter, coding is a clear step up, and the creative range surprised us. We're super excited about running a hackathon with Kimi on creativity. Kimi team continues to beat expectations!

K2.6 offers SOTA-level performance at a fraction of the cost. It's tremendously good at long-context tasks across the codebase, as well as the day-to-day work needed to support an always-on agent like KiloClaw.

Kimi K2.6 raises the bar for open-source models. It excels in coding and especially for agentic tools like OpenClaw and Hermes. In early testing, it sustains long multi-step sessions with impressive stability. It will work all of Ollama's integrations out of the box, and we're excited to see what developers build with it.

Within OpenCode, Kimi K2.6 proves to be exceptionally reliable. Its approach to task decomposition and tool calling is both steady and consistent. With a sharper grasp of task requirements and more streamlined multi-step operations, it effectively minimizes repetitive overhead, resulting in a smoother, more trustworthy end-to-end experience.

Kimi K2.6 delivered a strong performance in Qoder's internal evaluations, showing significant progress over K2.5. Specifically, there has been a notable increase in the frequency of tool calling and model invocations, reflecting a substantial boost in the model's proactivity and intelligence during task execution. This heightened initiative in tool calling enables the model to more actively grasp developer intent and automatically complete context, thereby minimizing user interruptions and wait times.

K2.6 shows major gains over K2.5 on the capabilities our developers care about most: we're seeing more than 50% improvement on our Next.js benchmark, putting it among the top-performing models on the platform. Combined with its cost-performance ratio, it's a compelling option for agentic coding and front-end generation through AI Gateway. We're excited to offer it to our developer community.

01 / 07

Coding-Driven Design

Based on the strong coding capabilities, Kimi K2.6 can turn simple prompts into complete front-end interfaces, generating structured layouts with deliberate design choices such as aesthetic hero sections, as well as interactive elements and rich animations, including scroll-triggered effects. With strong proficiency in leveraging image and video generation tools, Kimi K2.6 supports the generation of visually coherent assets and contributes to higher-quality, more salient hero sections.

Moreover, Kimi K2.6 expands beyond static frontend development to simple full-stack workflows—spanning authentication to user interaction to database operations for lightweight use cases like transaction logging or session management.

We established an internal Kimi Design Bench, organized into four categories: Visual Input Tasks, Landing Page Construction, Full-Stack Application Development, and General Creative Programming. In comparison with Google AI Studio, Kimi K2.6 shows promising results and performs well across these categories.

Below are examples generated by K2.6 Agent from a single prompt, with preconfigured harnesses and tools:

Agent Swarms, Elevated

Scaling out, not just up. An Agent Swarm dynamically decomposes tasks into heterogeneous subtasks executed concurrently by self-created domain-specialized agents.

Based on the K2.5 Agent Swarm research preview, Kimi K2.6 Agent Swarm demonstrates a qualitative leap in the agent swarm experience. It seamlessly coordinates heterogeneous agents to combine complementary skills: broad search layered with deep research, large-scale document analysis fused with long-form writing, and multi-format content generation executed in parallel. This compositional intelligence enables the swarm to deliver end-to-end outputs—spanning documents, websites, slides, and spreadsheets—within a single autonomous run.

The architecture scales horizontally to 300 sub-agents executing across 4,000 coordinated steps simultaneously, a substantial expansion from K2.5's 100 sub-agents and 1,500 steps. This massive parallelization fundamentally reduces end-to-end latency while significantly enhancing output quality and expanding the operational boundaries of Agents swarms.

It can also turn any high-quality files such as PDFs, spreadsheets, slides, and Word documents into Skills. Kimi K2.6 captures and maintains the documents' structural and stylistic DNA, enabling you to reproduce the same quality and format in future tasks.

Here are some examples:

Proactive Agents

K2.6 demonstrates strong performance in autonomous, proactive agents such as OpenClaw and Hermes, which operate across multiple applications with continuous, 24/7 execution.

Unlike simple chat-based interactions, these workflows require AI to proactively manage schedules, execute code, and orchestrate cross-platform operations as a persistent background agent.

Our RL infra team used a K2.6-backed agent that operated autonomously for 5 days, managing monitoring, incident response, and system operations, demonstrating persistent context, multi-threaded task handling, and full-cycle execution from alert to resolution. Here is K2.6's worklog (anonymized to remove sensitive information):

Kimi K2.6 delivers measurable improvements in real-world reliability: more precise API interpretation, stabler long-running performance, and enhanced safety awareness during extended research tasks.

Performance gains are quantified by our internal Claw Bench, the evaluation suite spanning five domains: Coding Tasks, IM Ecosystem Integration, Information Research & Analysis, Scheduled Task Management, and Memory Utilization. Across all metrics, Kimi K2.6 significantly outperforms Kimi K2.5 in task completion rates and tool invocation accuracy—particularly in workflows requiring sustained autonomous operation without human oversight.

Bring Your Own Agents

Building upon Kimi K2.6's robust orchestration capabilities, Kimi K2.6 extends your proactive agents to Claw Groups as a research preview—a new instantiation of the Agent Swarm architecture.

Claw Groups embrace an open, heterogeneous ecosystem: Multiple agents and humans operate as true collaborators. Users can onboard agents from any device, running any model, each carrying their own specialized toolkits, skills and persistent memory contexts. Whether deployed on local laptops, mobile devices, or cloud instances, these diverse agents integrate seamlessly into a shared operational space.

At the center of this swarm, Kimi K2.6 serves as an adaptive coordinator. It dynamically matches tasks to agents based on their specific skill profiles and available tools, optimizing for capability fit. When an agent encounters failure or stalls, the coordinator detects the interruption, automatically reassigns the task or regenerates subtasks, and actively manages the full lifecycle of deliverables—from initiation through validation to completion.

We also want to thank the K2.6-powered agents in Claw Groups—we've been dogfooding our own agent marketing team by refining human–agent workflows in practice. Using Claw Groups, we run end-to-end content production and launch campaigns, with specialized agents like Demo Makers, Benchmark Makers, Social Media Agents, and Video Makers working together. K2.6 coordinates the process, enabling agents to share intermediate results and turn ideas into consistent, fully packaged deliverables.

We are moving beyond simply asking AI a question or assigning AI a task, and entering a phase where human and AI collaborate as genuine partners—combining strengths to solve problems collectively. Claw Groups marks our latest efforts toward a future where the boundaries between "my agent," "your agent," and "our team" dissolve seamlessly into a collaborative system.

Benchmark Table

To reproduce official Kimi-K2.6 benchmark results, we recommend using the official API. For third-party providers, refer to Kimi Vendor Verifier (KVV) to choose high-accuracy services. Details: https://www.kimi.com/blog/kimi-vendor-verifier

Footnotes

1. General Testing Details

We report results for Kimi K2.6 and Kimi K2.5 with thinking mode enabled, Claude Opus 4.6 with max effort, GPT-5.4 with xhigh reasoning effort, and Gemini 3.1 Pro with a high thinking level.
Unless otherwise specified, all Kimi K2.6 experiments were conducted with temperature = 1.0, top-p = 1.0, and a context length of 262,144 tokens.
Benchmarks without publicly available scores were re-evaluated under the same conditions used for Kimi K2.6 and are marked with an asterisk (*). Except where noted with an asterisk, all other results are cited from official reports.

2. Reasoning Benchmarks

IMO-AnswerBench scores for GPT-5.4 and Claude 4.6 were obtained from https://z.ai/blog/glm-5.1.
Humanity's Last Exam (HLE) and other reasoning tasks were evaluated with a maximum generation length of 98,304 tokens. By default, we report results on the HLE full set. For the text-only subset, Kimi K2.6 achieves 36.4% accuracy without tools and 55.5% with tools.

3. Tool-Augmented / Agentic Tasks

Kimi K2.6 was equipped with search, code-interpreter, and web-browsing tools for HLE with tools, BrowseComp, DeepSearchQA, and WideSearch.
For HLE-Full with tools, the maximum generation length is 262,144 tokens with a per-step limit of 49,152 tokens. We employ a simple context management strategy: once the context window exceeds the threshold, only the most recent round of tool-related messages is retained.
For BrowseComp, we report scores obtained with context management using the same discard-all strategy as Kimi K2.5 and DeepSeek-V3.2.
For DeepSearchQA, no context management was applied to Kimi K2.6 tests, and tasks exceeding the supported context length were directly counted as failed. Scores for Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro on DeepSearchQA are cited from the Claude Opus 4.7 System Card.
For WideSearch, we report results under the "hide tool result" context management setting. Once the context window exceeds the threshold, only the most recent round of tool-related messages is retained.
The test system prompts are identical to those used in the Kimi K2.5 technical report.
Claw Eval was conducted using version 1.1 with max-tokens-per-step = 16384.
For APEX-Agents, we evaluate 452 tasks from the public 480-task release, as done by Artificial Analysis (excluding Investment Banking Worlds 244 and 246, which have external runtime dependencies).

4. Coding Tasks

Terminal-Bench 2.0 scores were obtained with the default agent framework (Terminus-2) and the provided JSON parser, operating in preserve thinking mode.
For the SWE-Bench series of evaluations (including Verified, Multilingual, and Pro), we used an in-house evaluation framework adapted from SWE-agent. This framework includes a minimal set of tools—bash tool, createfile tool, insert tool, view tool, strreplace tool, and submit tool.
All reported scores for coding tasks are averaged over 10 independent runs.

5. Vision Benchmarks

Max-tokens = 98,304, averaged over three runs (avg@3).
Settings with Python tool use max-tokens-per-step = 65,536 and max-steps = 50 for multi-step reasoning.
MMMU-Pro follows the official protocol, preserving input order and prepending images.

Agent Swarm

2026-02-09T00:00:00+00:00

In 2025, if you walked into any AI conference, you may hear the same gospel: faster inference, longer context windows, cheaper inference costs.

It's as if we've spent years perfecting the hammer, making it lighter, stronger, more precisely balanced, while never questioning the fact that the carpenter still has only two hands and twenty-four hours in a day.

Now, Kimi introduces Agent Swarm. It is not a better hammer. It is a reconstruction of the entire workshop.

This is not the story of "many AI agents working together". What we're building is an organizational structure with bosses, employees, and divisions of labor, except this organization isn't designed by humans. It designs itself.

When you tell Agent Swarm to research a topic, you aren't commanding an assistant. You are hiring a CEO, who helps find researchers, analysts, fact-checkers, all hired on the spot on its own. And you don't need to micromanage.

But to get there, we had to confront a question that we used to avoid:

The Limits of Single-Agent Reasoning

How far can we really push a model at test time?

This question isn't really about intelligence. The real bottleneck in AI reasoning today is the single-agent, sequential execution model itself.

Whether you're trying to extend the length of tasks a model can handle (what researchers call long-horizon tasks), or dynamically scaling compute and structure during inference (test-time scaling), the single-agent sequential execution model hits a wall.

For example, ask a single-agent deep research tool to survey a hundred companies or synthesize dozens of papers. As the task goes on, the context window fills. The system falls back to simple history folding or summarization to make room for new tokens. This compression is lossy, and later reasoning degrades.

It's not a bug, nor a temporary limitation. It's a structural ceiling imposed by context windows, time, and the constraints of relying on a single agent to manage long-horizon reasoning.

Scale Out, Not Just Up

For many years, the dominant narrative in AI has been scaling: bigger models, more parameters, better benchmark scores. This narrative has delivered remarkable results. But vertical scaling has a ceiling—physical, economic, and perhaps intellectual.

Horizontal scaling is different. One brain versus many. A single model is one expert. A self-organizing network is a company, a laboratory, an intelligence agency.

In June 2025, shortly after we launched Kimi Researcher, a member of our team — let's call her an "enthusiastic amateur stock trader"—had an idea that seemed simple enough: Could Kimi automatically gather daily information about stocks?

She let Kimi check news for macro trends. Query historical limit-up counts. Drill deeper if conditions match. Synthesize. She wrote a Python workflow, a cascade of if-else statements.

At line one hundred, she stopped. "I'm hand-coding a multi-agent system."

If models can use tools and handle long-horizon tasks, why can't they architect themselves? Decide when to parallelize, whom to hire, how to delegate?

The future isn't better single agents. It's agents that build organizations. Agent Swarm was born from this hypothesis.

The numbers tell part of the story. With Agent Swarm, Kimi K2.5 can:

Deploy up to 100 sub-agents working in parallel
Execute over 1,500 tool calls
Deliver better results 4.5x faster than sequential execution

Agent Swarm Best Practices

Obviously, Agent Swarm excels where work parallelizes: broad research, batch downloads, multi-file processing, multi-angle analysis, long-form writing.

But the deeper benefit is structural, it creates the conditions for productive disagreement — for independent agents to arrive at different conclusions, then force a reconciliation. It avoids groupthink structurally.

So in the following cases, we suggest you experience the magic of agent swarm.

Discovery at Scale

Start with the simplest case: finding a mountain of things that are hard to find.

Say you need the top 3 creators in 100 niche YouTube domains. K2.5 Agent Swarm first researches and defines each domain, then autonomously creates 100 sub-agents to conduct parallel searches.

Or you want to collect all 200+ Paul Graham essays, scattered across personal sites, old blogs, transcribed talks. K2.5 Agent Swarm can assign specialized sub-agents to search for, download, categorize, summarize, and compile Paul Graham's essays. They collectively organize over 200 original essays into 6 topic-based folders and produces a comprehensive summary report.

100 Sub-agents Hunting for Creators: https://www.kimi.ai/share/19c40eea-b272-8ef2-8000-0000af5e0baa

Output at Scale

Beyond gathering scattered information, you can task Agent Swarm with consuming massive document sets and coordinating expert personas to produce book-length, professional-grade reports.

For example, let it generate a 100-page literature review from forty social psychology PDFs. K2.5 Agent Swarm decomposes the task across the document set, deploying multiple writing-focused sub-agents. Each claims responsibility for specific sections, and their outputs are synthesized into a 100-page, two-column academic document with fully formatted citations and references.

Generate 100-Page Literature Review from 40 PDFs: https://www.kimi.ai/share/19c4106b-89b2-8361-8000-0000d07b8235

Perspective at Scale

The most interesting use case is when you need the disagreement itself—when you want to see a problem through multiple perspectives at once.

Facing a complex product launch? Deploy a team of experts: the skeptical VC questions unit economics, the veteran PM worries about technical debt, the ethicist probes dark patterns, customer success lead champions edge cases.

Get your product plan reviewed by a team of experts: https://www.kimi.ai/share/19c40bc9-31a2-8533-8000-0000bad59b7a

Or explore different story directions. You can have 20 writers from different literary styles to continue Liu Cixin's The Three-Body Problem: from Virginia Woolf–style inner monologues to Borges-like idea mazes, Kafkaesque absurd worlds, and Gabriel García Márquez–style stories shaped by repeating fate.

Let 20 writers create alternative endings for The Three-Body Problem: https://www.kimi.ai/share/19c409c8-8692-821a-8000-0000070ad369

Now Available to Top Tier Subscribers

You once had Kimi Agent as a single, diligent researcher. You now have Kimi Agent Swarm as a team of experts: specialized, parallel, capable of holding contradictory viewpoints simultaneously.

This is an early research preview. We will continue to harden the architecture, introducing direct sub-agent communication, dynamic control of parallel width—but the foundation is ready for your most demanding work.

In the AI age, literacy may be measured by how many tokens we use.

So type your prompt, and let Kimi self-direct 100 sub-agents for you: https://www.kimi.com/agent-swarm

WorldVQA

2026-02-03T00:00:00+00:00

Introducing WorldVQA

A benchmark for evaluating atomic visual world knowledge in Multimodal LLMs.

Authors Kimi Team

Overview

We are releasing WorldVQA, a new benchmark designed to measure the factual correctness of Multimodal Large Language Models (MLLMs). While recent models have demonstrated impressive capabilities in visual reasoning and description, measuring their reliability regarding visual world knowledge remains a challenge.

WorldVQA focuses on a critical question: Does the model actually recognize the specific entity it sees, or is it merely hallucinating based on visual patterns?

Our results show that WorldVQA creates a significant challenge for frontier models. Even state-of-the-art models struggle to achieve high accuracy on long-tail visual knowledge, often falling below 50% accuracy. This benchmark aims to drive progress toward more factually reliable and knowledgeable multimodal AI.

The Dataset

The dataset consists of 3,500 high-quality image-question pairs. The distribution aims to test a model's encyclopedic breadth across the world. The dataset distinguishes itself through three core design principles:

Factuality & Unambiguity: Every question has a single, verifiable ground-truth answer. We exclude subjective questions or ambiguous visual scenarios.
Rich Taxonomy: The dataset spans 9 categories to ensure broad coverage of world knowledge.
Head vs. Tail Distribution: We explicitly separate data into Head (common knowledge) and Tail (rare/long-tail knowledge). This allows us to measure how model performance degrades as knowledge becomes more obscure.

Note on Quality: To ensure the benchmark is a reliable gold standard, all images and question-answer pairs underwent rigorous multi-stage human verification to filter out noise and ambiguity.

Distribution of Tasks per Category

Statistics	Number
Data
Total	3500
Chinese (CN)	1260 (36%)
English (EN)	2240 (64%)
Category Categories
Total categories	9
Nature & Environment (Nature)	9.31%
Locations & Architecture (Geography)	14.63%
Culture, Arts & Crafts (Culture)	14.46%
Objects & Products (Objects)	12.49%
Vehicles, Craft & Transportation (Transportation)	8.74%
Entertainment, Media & Gaming (Entertainment)	14.60%
Brands, Logos & Graphic Design (Brands)	7.43%
Sports, Gear & Venues (Sports)	4.06%
Notable People & Public Figures (People)	14.29%
Difficulty
Easy	31.16%
Medium	40.77%
Hard	28.07%

Using WorldVQA to compare models

Measuring Calibration: Confidence vs. Accuracy

In our experiments comparing model confidence with actual accuracy, we utilized two key metrics to measure the alignment between a model's subjective belief and its objective performance:

ECE (Expected Calibration Error): Measures the average gap between the model's subjective confidence and its objective accuracy. The ideal value is 0.
Slope (Weighted Average Slope): Measures the correlation and sensitivity between the model's accuracy and its own confidence. The ideal value is 1.0.

Calibration and Confidence Distribution Analysis. Left: Reliability diagrams plotting Actual Accuracy against Stated Confidence. To ensure statistical significance, only bins containing more than 20 samples are visualized. The size of each data point is proportional to the number of samples in that bin. The black dashed diagonal (y=x) represents perfect calibration, while colored dashed lines indicate the weighted average slope for each model. Right: The distribution of stated confidence scores across the full dataset (without sample thresholding). The plots reveal a severe overconfidence trend, with most models concentrating their predictions in the 90-100% confidence range.

Our experiments reveal that all evaluated models are currently far from the ideal state, exhibiting a universal tendency toward overconfidence.

While Kimi-K2.5 achieves best performance on both metrics—recording an ECE of 37.9% and a Slope of 0.550—there remains a significant gap to bridge in the pursuit of "honesty" and "alignment." Enhancing the self-awareness boundaries of multimodal models represents a critical direction for future exploration.

Conclusion

WorldVQA is a simple but challenging benchmark for evaluating the atomic visual knowledge of frontier models. Improving performance on WorldVQA is a necessary step for the next generation of AI agents. We are open-sourcing the WorldVQA dataset and evaluation scripts to help the community address the visual knowledge gap.

Read the Paper: https://arxiv.org/abs/2602.02537
View the Code: https://github.com/MoonshotAI/WorldVQA
Download the Data: https://huggingface.co/datasets/moonshotai/WorldVQA

Kimi K2.5

2026-01-27T00:00:00+00:00

Today, we are introducing Kimi K2.5, the most powerful open-source model to date.

Kimi K2.5 builds on Kimi K2 with continued pretraining over approximately 15T mixed visual and text tokens. Built as a native multimodal model, K2.5 delivers state-of-the-art coding and vision capabilities and a self-directed agent swarm paradigm.

For complex tasks, Kimi K2.5 can self-direct an agent swarm with up to 100 sub-agents, executing parallel workflows across up to 1,500 tool calls. Compared with a single-agent setup, this reduces execution time by up to 4.5x. The agent swarm is automatically created and orchestrated by Kimi K2.5 without any predefined subagents or workflow.

Kimi K2.5 is available via Kimi.com, the Kimi App, the API, and Kimi Code. Kimi.com & Kimi App now supports 4 modes: K2.5 Instant, K2.5 Thinking, K2.5 Agent, and K2.5 Agent Swarm (Beta). Agent Swarm is currently in beta on Kimi.com, with free credits available for high-tier paid users.

Across three agentic benchmarks—HLE, BrowseComp, and SWE-Verified—Kimi K2.5 delivers strong performance at a fraction of the cost.

1. Coding with Vision

Kimi K2.5 is the strongest open-source model to date for coding, with particularly strong capabilities in front-end development.

K2.5 can turn simple conversations into complete front-end interfaces, implementing interactive layouts and rich animations such as scroll-triggered effects. Below are examples generated by K2.5 from a single prompt with image-gen tool:

Beyond text prompts, K2.5 excels at coding with vision. By reasoning over images and video, K2.5 improves image/video-to-code generation and visual debugging, lowering the barrier for users to express intent visually.

Here is an example of K2.5 reconstructing a website from video:

This capability stems from massive-scale vision-text joint pre-training. At scale, the trade-off between vision and text capabilities disappears — they improve in unison.

Below is an example of K2.5 reasoning over a puzzle and marking the shortest path using code:

K2.5 excels in real-world software engineering tasks. We evaluate it using Kimi Code Bench, our internal coding benchmark covering diverse end-to-end tasks — from building to debugging, refactoring, testing, and scripting — across multiple programming languages. On this benchmark, K2.5 shows consistent and meaningful improvements over K2 across task types.

To try out K2.5's agentic coding capabilities, K2.5 Agent offers a set of preconfigured tools for immediate, hands-on experiences. For software engineering use cases, we recommend pairing Kimi K2.5 with our new coding product, Kimi Code.

Kimi Code works in your terminal and can be integrated with various IDEs including VSCode, Cursor, Zed, etc. Kimi Code is open-sourced and supports images and videos as inputs. It also automatically discovers and migrates existing skills and MCPs into your working environment in Kimi Code.

Here's an example using Kimi Code to translate the aesthetic of Matisse's La Danse into the Kimi App. This demo highlights a breakthrough in autonomous visual debugging. Using visual inputs and documentation lookup, K2.5 visually inspects its own output and iterates on it autonomously. It creates an art-inspired webpage created end to end:

2. Agent Swarm

Scaling Out, Not Just Up. We release K2.5 Agent Swarm as a research preview, marking a shift from single-agent scaling to self-directed, coordinated swarm-like execution.

Trained with Parallel-Agent Reinforcement Learning (PARL), K2.5 learns to self-direct an agent swarm of up to 100 sub-agents, executing parallel workflows across up to 1,500 coordinated steps, without predefined roles or hand-crafted workflows.

PARL uses a trainable orchestrator agent to decompose tasks into parallelizable subtasks, each executed by dynamically instantiated, frozen subagents. Running these subtasks concurrently significantly reduces end-to-end latency compared to sequential agent execution.

Training a reliable parallel orchestrator is challenging due to delayed, sparse, and non-stationary feedback from independently running subagents. A common failure mode is serial collapse, where the orchestrator defaults to single-agent execution despite having parallel capacity. To address this, PARL employs staged reward shaping that encourages parallelism early in training and gradually shifts focus toward task success.

We define the reward as

$r_{\mathrm{PARL}}(x, y) = \lambda_1 \cdot \mkern-26mu \underbrace{r_{\text{parallel}}}_{\text{instantiation reward}} \mkern-9mu + \mkern18mu\lambda_2 \cdot \mkern-32mu\underbrace{r_{\text{finish}}}_{\text{sub-agent finish rate}} + \underbrace{r_{\text{perf}}(x, y)}_{\text{task-level outcome}} , .$

The performance reward r_(perf) evaluates the overall success and quality of the solution y for a given task x. This is augmented by two auxiliary rewards, each addressing a distinct challenge in learning parallel orchestration. The reward rₚₐᵣₐₗₗₑₗ is introduced to mitigate serial collapse—a local optimum where the orchestrator defaults to single-agent execution. By incentivizing subagent instantiation, this term encourages the exploration of concurrent scheduling spaces. The r_(finish) reward focuses on the successful completion of assigned subtasks. It is used to prevent spurious parallelism, a reward-hacking behavior in which the orchestrator increases parallel metrics dramatically by spawning many subagents without meaningful task decomposition. By rewarding completed subtasks, r_(finish) enforces feasibility and guides the policy toward valid and effective decompositions. The hyperparameters λ₁ and λ₂ are annealed to zero over the course of training.

To further force parallel strategies to emerge, we introduce a computational bottleneck that makes sequential execution impractical. Instead of counting total steps, we evaluate performance using Critical Steps, a latency-oriented metric inspired by the critical path in parallel computation:

CriticalSteps = ∑ₜ₌₁ᵀ ( Sₘₐᵢₙ⁽ᵗ⁾ + maxᵢ S_(sub,i)⁽ᵗ⁾ )

Sₘₐᵢₙ⁽ᵗ⁾captures orchestration overhead, while maxᵢ S_(sub,i)⁽ᵗ⁾ reflects the slowest subagent at each stage. Under this metric, spawning more subtasks only helps if it shortens the critical path.

An agent swarm has an orchestrator that dynamically creates specialized subagents (e.g., AI Researcher, Physics Researcher, Fact Checker) and decomposes complex tasks into parallelizable subtasks for efficient distributed execution.

In our parallel-agent reinforcement learning environment, the reward increases smoothly as training progresses. At the same time, the level of parallelism during training also gradually increases.

K2.5 Agent Swarm improves performance on complex tasks through parallel, specialized execution. In our internal evaluations, it leads to an 80% reduction in end-to-end runtime while enabling more complex, long-horizon workloads, as shown below.

Agent Swarm reduces the minimum critical steps required to achieve target performance by 3×–4.5× compared to single-agent execution in wide search scenario, with savings scaling as targets rise—translating to up to 4.5× wall-clock time reduction via parallelization.

Here are representative trajectories demonstrating K2.5 Agent Swarm in action:

3. Office Productivity

Kimi K2.5 brings agentic intelligence into real-world knowledge work.

K2.5 Agent can handle high-density, large-scale office work end to end. It reasons over large, high-density inputs, coordinates multi-step tool use, and delivers expert-level outputs: documents, spreadsheets, PDFs, and slide decks—directly through conversation.

With a focus on real-world professional tasks, we design two internal expert productivity benchmarks. The AI Office Benchmark evaluates end-to-end Office output quality, while the General Agent Benchmark measures multi-step, production-grade workflows against human expert performance. Across both benchmarks, K2.5 shows 59.3% and 24.3% improvements over K2 Thinking, reflecting stronger end-to-end performance on real-world tasks.

Internal Expert Productivity Bench (AI Office)

K2.5 agent supports advanced tasks such as adding annotations in Word, constructing financial models with Pivot Tables, and writing LaTeX equations in PDFs, while scaling to long-form outputs like 10,000-word papers or 100-page documents.

Tasks that once took hours or days now complete in minutes. Here are some examples:

4. Conclusion

Grounded in advances in coding with vision, agent swarms, and office productivity, Kimi K2.5 represents a meaningful step toward AGI for the open-source community, demonstrating strong capability on real-world tasks under real-world constraints. Looking ahead, we will push further into the frontier of agentic intelligence, redefining the boundaries of AI in knowledge work.

Appendix

Benchmark table

To reproduce official Kimi-K2.5 benchmark results, we recommend using the official API. For third-party providers, refer to Kimi Vendor Verifier (KVV) to choose high-accuracy services. Details: https://www.kimi.com/blog/kimi-vendor-verifier

Footnotes

1. General Testing Details

We report results for Kimi K2.5 and DeepSeek-V3.2 with thinking mode enabled, Claude Opus 4.5 with extended thinking mode, GPT-5.2 with xhigh reasoning effort, and Gemini 3 Pro with a high thinking level. For vision benchmarks, we additionally report results for Qwen3-VL-235B-A22B-Thinking.
Unless otherwise specified, all Kimi K2.5 experiments were conducted with temperature = 1.0, top-p = 0.95, and a context length of 256k tokens.
Benchmarks without publicly available scores were re-evaluated under the same conditions used for Kimi K2.5 and are marked with an asterisk (*).
We could not evaluate GPT-5.2 xhigh on all benchmarks due to service stability issues. For benchmarks that were not tested, we mark them as "-".

2. Text and Reasoning

HLE, AIME 2025, HMMT 2025 (Feb), GPQA-Diamond and IMO-AnswerBench were evaluated with a maximum completion budget of 96k tokens.
Results for AIME and HMMT are averaged over 32 runs (avg@32); GPQA-Diamond over 8 runs (avg@8).
For HLE, we report scores on the full set (text & image). Kimi K2.5 scores 31.5 (text) and 21.3 (image) without tools, and 51.8 (text) and 39.8 (image) with tools. The DeepSeek-V3.2 score corresponds to its text-only subset (marked with †) . Hugging Face access was blocked to prevent potential data leakage. HLE with tools uses simple context management: once the context exceeds a threshold, only the latest round of tool messages is retained.

3. Tool-Augmented / Agentic Search

Kimi K2.5 was equipped with search, code-interpreter, and web-browsing tools for HLE with tools and all agentic search benchmarks.
Except for BrowseComp (where K2.5 and DeepSeek-V3.2 used the discard-all strategy), no context management was applied, and tasks exceeding the supported context length were directly counted as failed.
The test system prompts emphasize deep and proactive tool use, instructing models to reason carefully, leverage tools, and verify uncertain information. Full prompts will be provided in the technical report.
Results for Seal-0 and WideSearch are averaged over four runs (avg@4).

4. Vision Benchmarks

Max-tokens = 64k, averaged over three runs (avg@3).
ZeroBench (w/ tools) uses max-tokens-per-step = 24k and max-steps = 30 for multi-step reasoning.
MMMU-Pro follows the official protocol, preserving input order and prepending images.
GPT-5.2-xhigh had ~10% failure rate (no output despite 3 retries), treated as incorrect; reported scores likely underestimate true performance.
WorldVQA, a benchmark designed to evaluate atomic vision-centric world knowledge. Access WorldVQA at https://github.com/MoonshotAI/WorldVQA.
OmniDocBench Score is computed as (1 − normalized Levenshtein distance) × 100, where a higher score denotes superior accuracy.

5. Coding Tasks

Terminal-Bench 2.0 scores were obtained with the default agent framework (Terminus-2) and the provided JSON parser. In our implementation, we evaluated Terminal-Bench 2.0 under non-thinking mode. This choice was made because our current context management strategy for the thinking mode is incompatible with Terminus-2.
For the SWE-Bench series of evaluations (including verified, multilingual, and pro), we used an internally developed evaluation framework. This framework includes a minimal set of tools—bash tool, createfile tool, insert tool, view tool, strreplace tool, and submit tool—along with tailored system prompts designed for the tasks. The highest scores were achieved under non-thinking mode.
The score of Claude Opus 4.5 on CyberGym is reported under the non-thinking setting.
All reported scores of coding tasks are averaged over 5 independent runs.

6. Long-Context Benchmarks

AA-LCR: scores averaged over three runs (avg@3).
LongBench-V2: identical prompts and input contexts standardized to ~128k tokens.

7. Agent Swarm

BrowseComp (Swarm Mode): main agent max 15 steps; sub-agents max 100 steps.
WideSearch (Swarm Mode): main and sub-agents max 100 steps.

Kimi Vendor Verifier

2026-01-22T00:00:00+00:00

Rebuilding the "Chain of Trust": Kimi Vendor Verifier

Alongside the release of the Kimi K2.6 model, we are open-sourcing the Kimi Vendor Verifier (KVV) project, designed to help users of open-source models verify the accuracy of their inference implementations.

Not as an afterthought, but because we learned the hard way that open-sourcing a model is only half the battle. The other half is ensuring it runs correctly everywhere else.

Official Evaluation Results

Think

Non-Think

Benchmark	Metric	Temperature	TopP	MaxTokens	Kimi API

OCRBench	acc	1.0	0.95	16384	91.0
AIME2025	avg@32	1.0	0.95	98304	98.4
MMMU Pro Vision	acc	1.0	0.95	65536	78.8

You can click here to access the Kimi API K2VV evaluation results for calculating the F1 score.

Why We Built KVV

From Isolated Incidents to Systemic Issues

Since the release of K2 Thinking, we have received frequent feedback from the community regarding anomalies in benchmark scores. Our investigation confirmed that a significant portion of these cases stemmed from the misuse of Decoding parameters. To mitigate this immediately, we built our first line of defense at the API level: enforcing Temperature=1.0 and TopP=0.95 in Thinking mode, with mandatory validation that thinking content is correctly passed back.

However, more subtle anomalies soon triggered our alarm. In a specific evaluation on LiveBenchmark, we observed a stark contrast between third-party API and official API. After extensive testing of various infrastructure providers, we found this difference is widespread.

This exposed a deeper problem in the open-source model ecosystem: The more open the weights are, and the more diverse the deployment channels become, the less controllable the quality becomes.

If users cannot distinguish between "model capability defects" and "engineering implementation deviations," trust in the open-source ecosystem will inevitably collapse.

Our Solution

Six Critical Benchmarks (selected to expose specific infra failures):

Pre-Verification: Validates that API parameter constraints (temperature, top_p, etc.) are correctly enforced. All tests must pass before proceeding to benchmark evaluation.
OCRBench: 5 minutes smoke test for multimodal pipelines.
MMMU Pro: Verify Vision input preprocessing by testing diverse visual inputs.
AIME2025: Long-output stress test. Catches KV cache bugs and quantization degradation that short benchmarks hide.
K2VV ToolCall: Measures trigger consistency (F1) and JSON Schema accuracy. Tool errors compound in agents; we catch them early.
SWE-Bench: Full agentic coding test. (Not open sourced due to dependency of sandbox)

Upstream Fix: We embed with vLLM/SGLang/KTransformers communities to fix root causes, not just detect symptoms.

Pre-Release Validation: Rather than waiting for post-deployment complaints, we provide early access to test models. This lets infrastructure providers validate their stacks before users encounter issues.

Continuous Benchmarking: We will maintain a public leaderboard of vendor results. This transparency encourages vendors to prioritize accuracy.

Testing Cost Estimation

We completed full evaluation workflow validation on Two NVIDIA H20 8-GPU servers, with sequential execution taking approximately 15 hours. To improve evaluation efficiency, scripts have been optimized for long-running inference scenarios, including streaming inference, automatic retry, and checkpoint resumption mechanisms.

An Open Invitation

Weights are open. The knowledge to run them correctly must be too.

We are expanding vendor coverage and seeking lighter agentic tests. Contact Us: contact-kvv@kimi.com

Kimi K2 Thinking

2025-11-06T00:00:00+00:00

Today, we are introducing Kimi K2 Thinking, our best open-source thinking model.

Built as a thinking agent, it reasons step by step while using tools, achieving state-of-the-art performance on Humanity's Last Exam (HLE), BrowseComp, and other benchmarks, with major gains in reasoning, agentic search, coding, writing, and general capabilities.

Kimi K2 Thinking can execute up to 200 – 300 sequential tool calls without human interference, reasoning coherently across hundreds of steps to solve complex problems.

It marks our latest efforts in test-time scaling, by scaling both thinking tokens and tool calling steps.

K2 Thinking is now live on kimi.com under the chat mode ^[1], with its full agentic mode available soon. It is also accessible through the Kimi K2 Thinking API.

Evaluations

Kimi K2 Thinking sets new records across benchmarks that assess reasoning, coding, and agent capabilities. K2 Thinking achieves 44.9% on HLE with tools, 60.2% on BrowseComp, and 71.3% on SWE-Bench Verified, demonstrating strong generalization as a state-of-the-art thinking agent model.

Agentic Reasoning

K2 Thinking demonstrates outstanding reasoning and problem-solving abilities. On Humanity’s Last Exam (HLE)—a rigorously crafted, closed‑ended benchmark—spanning thousands of expert‑level questions across more than 100 subjects, K2 Thinking achieved a state-of-the-art score of 44.9%, with search, python, and web-browsing tools, establishing new records in multi‑domain expert‑level reasoning performance.

By reasoning while actively using a diverse set of tools, K2 Thinking is capable of planning, reasoning, executing, and adapting across hundreds of steps to tackle some of the most challenging academic and analytical problems. In one instance, it successfully solved a PhD-level mathematics problem through 23 interleaved reasoning and tool calls, exemplifying its capacity for deep, structured reasoning and long-horizon problem solving:

Agentic Coding

K2 Thinking exhibits substantial gains in coding and software development tasks. It achieves scores of 61.1% on SWE-Multilingual, 71.3% on SWE-Bench Verified, and 47.1% on Terminal-Bench, showcasing strong generalization across programming languages and agent scaffolds.

The model delivers notable improvements on HTML, React, and component-intensive front-end tasks—translating ideas into fully functional, responsive products. In agentic coding settings, it reasons while invoking tools, integrating fluidly into software agents to execute complex, multi-step development workflows with precision and adaptability.

Here are some examples that Kimi K2 Thinking has built from a single prompt:

Agentic Search and Browsing

K2 Thinking demonstrates strong performance in agentic search and browsing scenarios. On BrowseComp—a challenging benchmark designed to evaluate models' ability to continuously browse, search, and reason over hard-to-find real-world web information—K2 Thinking achieved a score of 60.2%, significantly outperforming the human baseline of 29.2%. This result highlights K2 Thinking's superior capability for goal-directed, web-based reasoning and its robustness in dynamic, information-rich environments.

K2 Thinking can execute 200–300 sequential tool calls, driven by long-horizon planning and adaptive reasoning. It performs dynamic cycles of think → search → browser use → think → code, continually generating and refining hypotheses, verifying evidence, reasoning, and constructing coherent answers. This interleaved reasoning allows it to decompose ambiguous, open-ended problems into clear, actionable subtasks.

Here's one example of what Kimi K2 Thinking has researched from just a single prompt:

General Capabilities

Creative Writing: K2 Thinking delivers improvements in completeness and richness. It shows stronger command of style and instruction, handling diverse tones and formats with natural fluency. Its writing becomes more vivid and imaginative—poetic imagery carries deeper associations, while stories and scripts feel more human, emotional, and purposeful. The ideas it expresses often reach greater thematic depth and resonance.

Practical Writing: K2 Thinking demonstrates marked gains in reasoning depth, perspective breadth, and instruction adherence. It follows prompts with higher precision, addressing each requirement clearly and systematically—often expanding on every mentioned point to ensure thorough coverage. In academic, research, and long-form analytical writing, it excels at producing rigorous, logically coherent, and substantively rich content, making it particularly effective in scholarly and professional contexts.

Personal & Emotional: When addressing personal or emotional questions, K2 Thinking responds with more empathy and balance. Its reflections are thoughtful and specific, offering nuanced perspectives and actionable next steps. It helps users navigate complex decisions with clarity and care—grounded, practical, and genuinely human in tone.

Inference Efficiency

Low-bit quantization is an effective way to reduce inference latency and GPU memory usage on large-scale inference servers. However, thinking models use excessive decoding lengths, and thus quantization often results in substantial performance drops.

To overcome this challenge, we adopt Quantization-Aware Training (QAT) during the post-training phase, applying INT4 weight-only quantization to the MoE components. It allows K2 Thinking to support native INT4 inference with a roughly 2x generation speed improvement while achieving state-of-the-art performance. All benchmark results are reported under INT4 precision.

Full Evaluations ^[2]

The table below shows that Kimi K2 Thinking matches or surpasses the latest open-source and frontier models across a wide range of tasks, excelling on benchmarks for reasoning, agentic search, and coding.

Footnotes

To ensure a fast, lightweight experience, we selectively employ a subset of tools and reduce the number of tool call turns under the chat mode on kimi.com. As a result, chatting on kimi.com may not reproduce our benchmark scores. Our agentic mode will be updated soon to reflect the full capabilities of K2 Thinking.
Testing Details: a. All benchmarks were evaluated at temperature = 1.0 and 256 k context length for K2 Thinking, except for SciCode, for which we followed the official temperature setting of 0.0. b. HLE (no tools), AIME25, HMMT25, and GPQA were capped at a 96k thinking-token budget, while IMO-Answer Bench, LiveCodeBench and OJ-Bench were capped at a 128k thinking-token budget. Longform Writing was capped at a 32k completion-token budget. c. For AIME and HMMT (no tools), we report the average of 32 runs (avg@32). For AIME and HMMT (with Python), we report the average of 16 runs (avg@16). For IMO-AnswerBench, we report the average of 8 runs (avg@8).
Baselines: a. GPT-5, Claude-4.5-sonnet, Grok-4 results and DeepSeek-V3.2 results are quoted from the GPT-5 post, GPT-5 for Developers post, GPT-5 system card, claude-sonnet-4-5, grok-4, deepseek-v3.2, the public Terminal-Bench leaderboard (Terminus-2), the public Vals AI leaderboard and the artificialanalysis. Benchmarks for which no available public scores were re-tested under the same conditions used for k2 thinking and are marked with an asterisk(*). For the GPT-5 test, we set the reasoning effort to high. b. The GPT-5 and Grok-4 on the HLE full set with tools are 35.2 and 38.6 from their official posts. In our internal evaluation on the HLE text-only subset, GPT-5 scores 41.7 and Grok-4 scores 38.6 (Grok-4’s launch cited 41.0 on the text-only subset). For GPT-5's HLE text-only w/o tool, we use score from Scale.ai, and the official GPT-5 score on the HLE full set (no tools) is 24.8. c. For IMO-AnswerBench: GPT-5 scored 65.6 in the benchmark paper. We re-evaluated GPT-5 with official API and obtained a score of 76.
For HLE (w/ tools) and the agentic-search benchmarks: a. K2 Thinking was equipped with search, code-interpreter, and web-browsing tools. b. BrowseComp-ZH, Seal-0, FinSearchComp-T3 were run 4 times independently and the average is reported (avg@4). c. The evaluation used o3-mini as judge, configured identically to the official HLE setting; judge prompts were taken verbatim from the official repository. d. On HLE, the maximum step limit was 120, with a 48 k-token reasoning budget per step; on agentic-search tasks, the limit was 300 steps with a 24 k-token reasoning budget per step. e. When tool execution results cause the accumulated input to exceed the model's context limit (256k), we employ a simple context management strategy that hides all previous tool outputs. f. The web access to Hugging Face may lead to data leakage in certain benchmark tests, such as HLE. K2 Thinking can achieve a score of 51.3 on HLE without blocking Hugging Face. To ensure a fair and rigorous comparison, we blocked access to Hugging Face during testing.
For Coding Tasks: a. Terminal-Bench scores were obtained with the default agent framework (Terminus-2) and the provided JSON parser. b. For other coding tasks, the result was produced with our in-house evaluation harness. The harness is derived from SWE-agent, but we clamp the context windows of the Bash and Edit tools and rewrite the system prompt to match the task semantics. c. All reported scores of coding tasks are averaged over 5 independent runs.
Heavy Mode: K2 Thinking Heavy Mode employs an efficient parallel strategy: it first rolls out eight trajectories simultaneously, then reflectively aggregates all outputs to generate the final result. Heavy mode for GPT-5 denotes the official GPT-5 Pro score.

Kimi-K2-Instruct-0905

2025-09-05T00:00:00+00:00

📰 Tech Blog | 📄 Paper

1. Model Introduction

Kimi K2-Instruct-0905 is the latest, most capable version of Kimi K2. It is a state-of-the-art mixture-of-experts (MoE) language model, featuring 32 billion activated parameters and a total of 1 trillion parameters.

Key Features

Enhanced agentic coding intelligence: Kimi K2-Instruct-0905 demonstrates significant improvements in performance on public benchmarks and real-world coding agent tasks.
Improved frontend coding experience: Kimi K2-Instruct-0905 offers advancements in both the aesthetics and practicality of frontend programming.
Extended context length: Kimi K2-Instruct-0905’s context window has been increased from 128k to 256k tokens, providing better support for long-horizon tasks.

2. Model Summary


Architecture	Mixture-of-Experts (MoE)
Total Parameters	1T
Activated Parameters	32B
Number of Layers (Dense layer included)	61
Number of Dense Layers	1
Attention Hidden Dimension	7168
MoE Hidden Dimension (per Expert)	2048
Number of Attention Heads	64
Number of Experts	384
Selected Experts per Token	8
Number of Shared Experts	1
Vocabulary Size	160K
Context Length	256K
Attention Mechanism	MLA
Activation Function	SwiGLU

3. Evaluation Results

Benchmark	Metric	K2-Instruct-0905	K2-Instruct-0711	Qwen3-Coder-480B-A35B-Instruct	GLM-4.5	DeepSeek-V3.1	Claude-Sonnet-4	Claude-Opus-4
SWE-Bench verified	ACC	69.2 ± 0.63	65.8	69.6*	64.2*	66.0*	72.7*	72.5*
SWE-Bench Multilingual	ACC	55.9 ± 0.72	47.3	54.7*	52.7	54.5*	53.3*	-
Multi-SWE-Bench	ACC	33.5 ± 0.28	31.3	32.7	31.7	29.0	35.7	-
Terminal-Bench	ACC	44.5 ± 2.03	37.5	37.5*	39.9*	31.3*	36.4*	43.2*
SWE-Dev	ACC	66.6 ± 0.72	61.9	64.7	63.2	53.3	67.1	-

All K2-Instruct-0905 numbers are reported as mean ± std over five independent, full-test-set runs. Before each run we prune the repository so that every Git object unreachable from the target commit disappears; this guarantees the agent sees only the code that would legitimately be available at that point in history.

Except for Terminal-Bench (Terminus-2), every result was produced with our in-house evaluation harness. The harness is derived from SWE-agent, but we clamp the context windows of the Bash and Edit tools and rewrite the system prompt to match the task semantics. All baseline figures denoted with an asterisk (*) are excerpted directly from their official report or public leaderboard; the remaining metrics were evaluated by us under conditions identical to those used for K2-Instruct-0905.

For SWE-Dev we go one step further: we overwrite the original repository files and delete any test file that exercises the functions the agent is expected to generate, eliminating any indirect hints about the desired implementation.

4. Deployment

You can access Kimi K2's API on https://platform.moonshot.ai , we provide OpenAI/Anthropic-compatible API for you.

The Anthropic-compatible API maps temperature by real_temperature = request_temperature * 0.6 for better compatible with existing applications.

Our model checkpoints are stored in the block-fp8 format, you can find it on Huggingface.

Currently, Kimi-K2 is recommended to run on the following inference engines:

vLLM
SGLang
KTransformers
TensorRT-LLM

Deployment examples for vLLM and SGLang can be found in the Model Deployment Guide.

5. Model Usage

Chat Completion

Once the local inference service is up, you can interact with it through the chat endpoint:

def simple_chat(client: OpenAI, model_name: str):
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
        {"role": "user", "content": [{"type": "text", "text": "Please give a brief self-introduction."}]},
    ]
    response = client.chat.completions.create(
        model=model_name,
        messages=messages,
        stream=False,
        temperature=0.6,
        max_tokens=256
    )
    print(response.choices[0].message.content)

The recommended temperature for Kimi-K2-Instruct-0905 is temperature = 0.6. If no special instructions are required, the system prompt above is a good default.

Tool Calling

Kimi-K2-Instruct-0905 has strong tool-calling capabilities. To enable them, you need to pass the list of available tools in each request, then the model will autonomously decide when and how to invoke them.

The following example demonstrates calling a weather tool end-to-end:

# Your tool implementation
def get_weather(city: str) -> dict:
    return {"weather": "Sunny"}
# Tool schema definition
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Retrieve current weather information. Call this when the user asks about the weather.",
        "parameters": {
            "type": "object",
            "required": ["city"],
            "properties": {
                "city": {
                    "type": "string",
                    "description": "Name of the city"
                }
            }
        }
    }
}]
# Map tool names to their implementations
tool_map = {
    "get_weather": get_weather
}
def tool_call_with_client(client: OpenAI, model_name: str):
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
        {"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
    ]
    finish_reason = None
    while finish_reason is None or finish_reason == "tool_calls":
        completion = client.chat.completions.create(
            model=model_name,
            messages=messages,
            temperature=0.6,
            tools=tools,          # tool list defined above
            tool_choice="auto"
        )
        choice = completion.choices[0]
        finish_reason = choice.finish_reason
        if finish_reason == "tool_calls":
            messages.append(choice.message)
            for tool_call in choice.message.tool_calls:
                tool_call_name = tool_call.function.name
                tool_call_arguments = json.loads(tool_call.function.arguments)
                tool_function = tool_map[tool_call_name]
                tool_result = tool_function(**tool_call_arguments)
                print("tool_result:", tool_result)
                messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "name": tool_call_name,
                    "content": json.dumps(tool_result)
                })
    print("-" * 100)
    print(choice.message.content)

The tool_call_with_client function implements the pipeline from user query to tool execution. This pipeline requires the inference engine to support Kimi-K2’s native tool-parsing logic. For more information, see the Tool Calling Guide.

6. License

Both the code repository and the model weights are released under the Modified MIT License.

7. Third Party Notices

See THIRD PARTY NOTICES

7. Contact Us

If you have any questions, please reach out at support@moonshot.cn.

Kimi K2

2025-07-11T00:00:00+00:00

Update(0905)：We have an updated weight that enhances agentic coding and supports 256K context，see here

Kimi K2 is our latest Mixture-of-Experts model with 32 billion activated parameters and 1 trillion total parameters. It achieves state-of-the-art performance in frontier knowledge, math, and coding among non-thinking models. But it goes further — meticulously optimized for agentic tasks, Kimi K2 does not just answer; it acts.

And now, it is within your reach. Today, we are open-sourcing:

Kimi-K2-Base: The foundation model, a strong start for researchers and builders who want full control for fine-tuning and custom solutions.
Kimi-K2-Instruct: The post-trained model best for drop-in, general-purpose chat and agentic experiences. It is a reflex-grade model without long thinking.

With Kimi K2, advanced agentic intelligence is more open and accessible than ever. We can't wait to see what you build.

Use cases

Give Kimi K2 your tools and describe your task.

It automatically understands how to use the tools and gets the job done. You don't have to write any complex workflow for agentic applications.

Imagine using Kimi K2 to explore remote-work salaries with the Salary Data Analysis example, where 16 IPython calls generate stats, visualizations, and an interactive webpage of insights. Dive into the Stanford NLP Genealogy, and get Kimi K2 to generate an interactive site built through 5 web searches, 4 browsings, 3 clicks, 5 scrolls, 6 edits, and 2 deployments. Or plan your dream Coldplay Tour 2025 in London with Kimi K2, it crafted the plan for you through 17 seamless tool calls spanning search, calendar, Gmail, flights, Airbnb, and restaurant bookings.

Bring Kimi K2 to your command line. It edits files. It runs commands.

Kimi K2 understands your environment, decides what actions to take, and executes them seamlessly.

For clarity, the terminal displays only a summary of each trajectory. In every example, Kimi K2 orchestrates multiple tools and commands behind the scenes to accomplish complex objectives. For instance, Kimi K2 can automate Minecraft development in JavaScript: it manages rendering, runs and debugs test cases, captures logs on failure, and iteratively improves the code until all tests succeed. For pre/post norm analysis, Kimi K2 uses the Weights & Biases (wandb) data reader to extract insights from language model experiments and generates a polished analysis report. When converting a Flask project to Rust, Kimi K2 systematically refactors the codebase and runs performance benchmarks to ensure robust results.

Benchmarking Kimi K2

Open Agentic Intelligence

Pre-training is the crucial foundation for Agentic Intelligence, establishing the priors that makes reinforcement learning (RL) exploration tractable, efficient, and generalizable. However, as Ilya Sutskever also observes, human data is a finite "fossil fuel", and its growth is lagging far behind the pace of compute. This makes token efficiency during pre-training a new critical coefficient in the AI scaling laws.

Post-training is pivotal in the "Era of Experience" (David Silver, Richard Sutton, 2025). In this era, LLMs increasingly learn from their own self-generated interactions, receiving rewards that free them from the limits of human data and enable them to surpass human capabilities.

Kimi K2 is forged from these very insights.

MuonClip Optimizer

Without rigor, given an approximately finite pretraining dataset and a fixed model configuration, a more token-efficient optimizer generates more intelligence. Our previous work Moonlight has demonstrated that the Muon optimizer substantially outperforms the widely-used AdamW optimizer for LLM training.

Kimi K2 was designed to further scale up Moonlight, which employs an architecture similar to DeepSeek-V3. Based on scaling-law analysis, we reduce the number of heads for long-context efficiency, and increase MoE sparsity for greater token efficiency. While scaling up, we encountered a persistent challenge: training instability caused by exploding attention logits, an issue that occurs more frequently with Muon but less with AdamW in our experiments. Existing solutions such as logit soft-capping and query-key normalization were found inadequate.

To address this, we introduce the MuonClip optimizer that improves Muon with our proposed qk-clip technique. Specifically, qk-clip stabilizes training by directly rescaling the weight matrices of the query and key projections after Muon updates, thus controlling the scale of attention logits at the source. Concretely, the query and key projections are scaled as follows:

$qᵢ = η^α W_q xᵢ$

$kᵢ = η^(1-α) Wₖ xᵢ$

where α is a balancing hyperparameter, so the attention logit becomes:

$(η^α qᵢ)^⊤ (η^(1-α) kⱼ) = η, qᵢ^⊤ kⱼ$

The adaptive factor η (with threshold t) is set after every step based on the max attention logit in this step:

$η = min(t/( max_(i,j)(qᵢ^⊤ kⱼ)), 1)$

where t is a pre-set threshold. This is a general technique that can be possibly applied to other stabilization use cases.

Our experiments show that MuonClip effectively prevents logit explosions while maintaining downstream task performance. In practice, Kimi K2 was pre-trained on 15.5T tokens using MuonClip with zero training spike, demonstrating MuonClip as a robust solution for stable, large-scale LLM training.

Agentic Capabilities

The enhanced agentic capabilities of Kimi K2 originate from two important aspects — large-scale agentic data synthesis and general reinforcement learning.

Large-Scale Agentic Data Synthesis for Tool Use Learning: To teach the model sophisticated tool-use capabilities, we developed a comprehensive pipeline inspired by ACEBench that simulates real-world tool-using scenarios at scale. Our approach systematically evolves hundreds of domains containing thousands of tools—including both real MCP (Model Context Protocol) tools and synthetic ones—then generates hundreds of agents with diverse tool sets.

All tasks are rubric-based, enabling consistent evaluation. Agents interact with simulated environments and user agents, creating realistic multi-turn tool-use scenarios. An LLM judge evaluates simulation results against task rubrics, filtering for high-quality training data. This scalable pipeline generates diverse, high-quality data, paving the way for large-scale rejection sampling and reinforcement learning.

General Reinforcement Learning: The key challenge is to apply RL to tasks with both verifiable and non-verifiable rewards; typical examples of verifiable tasks are math and competition coding, while writing a research report is usually viewed as non-verifiable. Going beyond verifiable rewards, our general RL system uses a self-judging mechanism where the model acts as its own critic, providing scalable, rubric-based feedback for non-verifiable tasks.

Meanwhile, on-policy rollouts with verifiable rewards are used to continuously update the critic so that the critic keeps improving its evaluation accuracy on the latest policy. This can be viewed as a way of using verifiable rewards to improve the estimation of non-verifiable rewards.

Getting started with Kimi K2

Try Kimi K2 on kimi.com

Starting today, Kimi users on web and mobile can select and use the new Kimi K2 model for free. At this moment, our MCP features for web and app are still in development. We hope to begin rolling them out in the coming weeks. In the meantime, you’re welcome to try our Researcher for an early look at its agentic capabilities. Please note that vision features are not supported for Kimi K2 yet.

Use Kimi K2 with API

The Kimi Platform offers an OpenAI/Anthropic compatible interface, allowing for easy adaptation of your existing applications to Kimi K2. We encourage developers to explore our tool calling API for building agent applications. For detailed information, visit platform.kimi.com.

Serve Kimi K2 on your own

We recommend running Kimi K2 on one of the following inference engines: vLLM, SGLang, KTransformers, or TensorRT-LLM. For detailed deployment instructions, please see our GitHub repository.

What's next

While Kimi K2 serves as a strong foundation for open agentic intelligence, a general agent uses more advanced capabilities such as thinking and visual understanding. We plan to add these to Kimi K2 in the future.

Limitations

In our internal tests, we've identified some limitations in current Kimi K2 models. When dealing with hard reasoning tasks or unclear tool definition, the model may generate excessive tokens, sometimes leading to truncated outputs or incomplete tool calls. Additionally, performance may decline on certain tasks if tool use is enabled. When building complete software projects, one-shot prompting yields performance degradation compared to using K2 under an agentic framework. We are working to address these issues in future releases and looking forward to more feedbacks.

Kimi-Researcher

2025-06-20T00:00:00+00:00

End-to-End RL Training for Emerging Agentic Capabilities

June 20, 2025 • 10 min read

Meet Kimi-Researcher, an autonomous agent that excels at multi-turn search and reasoning. It performs an average of 23 reasoning steps and explores over 200 URLs per task. Built on an internal version of the Kimi k-series model and trained entirely through end-to-end agentic reinforcement learning (RL), it achieved a Pass@1 score of 26.9%—a state-of-the-art result—on Humanity's Last Exam, and Pass@4 accuracy of 40.17%. Starting from an initial HLE score of 8.6%, Kimi-Researcher reached 26.9% almost entirely through end-to-end RL training, providing compelling evidence that end-to-end agentic RL can significantly advance agent intelligence.

Kimi-Researcher has also achieved strong performance across several complex and challenging real-world benchmarks. On xbench, a new, dynamic, professionally-aligned suite designed to bridge AI capabilities with real-world productivity, Kimi-Researcher achieved 69% pass@1 (averaged on 4 runs) on xbench-DeepSearch, outperforming models such as o3 with search tools. On benchmark tests for multi-turn search reasoning (FRAMES, Seal-0) and factual information (SimpleQA), Kimi-Researcher also achieved strong performance.

Figure 1

Potential fluctuations in tools, such as search engines, may affect performance. The results are tested on: HLE on June 17, 2025; and xbench-DeepSearch, Seal-0, Frames, and SimpleQA on June 18, 2025.
All Kimi-Researcher results were evaluated using o3-mini. Scores of other models are referenced from the relevant papers or leaderboards. [1] [2] [3] [4] [5]
For benchmarks with fewer than 200 test samples (xbench, Seal-0), we performed four runs and reported the average result (avg@4).
We do not compare multi-agent workflows based on multiple frontier models here, as our focus is on evaluating model capabilities.

End-to-end agentic RL is promising but challenging

Kimi-Researcher is an autonomous agentic and thinking model designed to solve complex problems through multi-step planning, reasoning, and tool use. It leverages three main tools: a parallel, real-time internal search tool; a text-based browser tool for interactive web tasks; and a coding tool for automated code execution.

Formally, given the state observation $s_t$ (for instance, $s_0$ includes system prompt, tool declarations, and user query) , Kimi-Researcher generates $\text{think}_t$ and $\text{action}_t$. An action can either be a tool call or an indication to terminate the trajectory. The detailed behavior of Kimi-Researcher is as follows: \begin{cases} (s_t) \xrightarrow{\text{Kimi-Researcher}} (\text{think}_t, \text{action}_t) \\ s_{t+1} = \text{context_manager}(s_t, \text{think}_t, \text{tool_call_result}_t) & \text{if } \text{action}_t \neq \text{finish} \\ \text{terminate} & \text{if } \text{action}_t = \text{finish} \end{cases}

Traditional agent development has key limitations:

Workflow-Based Systems: Multi-agent workflows assign roles to specialized agents and coordinate the agents using prompt-based workflows. While effective, they are tied to specific LLM versions and need frequent manual updates as models or environments change, reducing scalability and flexibility.
Imitation Learning with Supervised Finetuning (SFT): Imitation learning aligns models well with human demonstrations but struggles with data labeling—especially for long-horizon, agentic tasks in dynamic environments. Furthermore, SFT datasets are tightly coupled with specific tool versions, resulting in poor generalization as tools evolve.

End-to-end agentic reinforcement learning trains a single model to solve problems holistically: given a query, the agent explores a large number of possible strategies, receives rewards for correct solutions, and learns from the full trajectory. Unlike SFT, it naturally handles long, on-policy reasoning and adapts to changing tools and environments; unlike modular approaches, all skills—planning, perception, and tool use—are learned together without hand-crafted rules or workflow templates. Previous work like OpenAI's Deep Research also highlights the strong performance of this approach, but it introduces new challenges:

Dynamic Environments: Agents must adapt to constantly changing conditions, as even identical queries can yield different results over time. The goal is robust generalization despite distribution shifts.
Long-Horizon Tasks: Kimi-Researcher can run 70+ search queries per trajectory, with context windows reaching hundreds of thousands of tokens. This demands advanced memory management and long-context models.
Data Scarcity: High-quality RL datasets for agentic QA are rare. We address this by automatically synthesizing training data, allowing large-scale learning without manual labeling.
Rollout Efficiency: Multi-turn reasoning and heavy tool use can slow training and cause GPU under-utilization. Optimizing rollout efficiency is crucial for scalable, practical agent RL training.

* calculated based on a small set of queries.

Approach

Kimi-Researcher is trained via end-to-end reinforcement learning. We observe a consistent improvement in agent performance across different domains. Figure 2-a illustrates the overall training accuracy of Kimi-Researcher throughout the reinforcement learning process. Figure 2-b presents model performance on several internal datasets.

Figure 2-a

Figure 2-b

Training data

To address the scarcity of high-quality agentic datasets, we engineered our training corpus with two complementary objectives.

First, we developed a suite of challenging, tool-centric tasks designed to promote robust tool-use learning. These prompts are deliberately constructed such that solving the task requires invoking specific tools—making naive approaches either infeasible or substantially less efficient. By embedding tool dependencies into task design, the agent learns not only when to invoke a tool, but also how to orchestrate tool use effectively in complex, real-world settings. (See Figure 3 for tool invocation rates using these training data.)

Figure 3

Figure 4

Second, we curated and synthesized reasoning-intensive tasks to reinforce the agent's core cognitive abilities and its capacity to integrate reasoning with tool usage. This component is further subdivided into:

Math and Code Reasoning: Tasks that target logical inference, algorithmic problem-solving, and sequential computation. Kimi-Researcher learns to solve this kind of problem with our toolset beyond purely using chain-of-thought.
Hard Search: Scenarios where the agent must iteratively search, synthesize, and reason within context constraints to derive valid answers. Case studies illustrate how these hard search tasks drive the emergence of deeper planning and robust, tool-augmented reasoning strategies.

To build this diverse prompt set at scale, we developed a fully automated pipeline capable of generating and validating many question-answer pairs with minimal manual intervention, ensuring both diversity and correctness at unprecedented scale. Ensuring accurate ground truth (GT) is critical for synthetic tasks, so we introduced a robust GT extraction method to guarantee that each question is paired with a reliable answer whenever possible. Additionally, a rigorous filtering funnel removes ambiguous, trivial, or incorrect pairs — with Pass@N checks ensuring only non-trivial questions are retained. Figure 4 shows the effectiveness of our synthetic tasks based on two experimental results.

RL training

The model is primarily trained using the REINFORCE algorithm. We have observed that the following factors contribute to more stable training:

On-policy Training: It is critical to generate strict on-policy data. During training, we disable LLM engine mechanisms like toolcall format enforcers to ensure each trajectory is generated entirely based on the model's own probability distribution.
Negative Sample Control: Negative samples lead to a decrease in token probabilities, which increases the risk of entropy collapse during RL training. To address this, we discard some negative samples strategically, allowing the model to continue improving over a longer training period.

Kimi-Researcher uses outcome rewards for training, aiming to provide a constant preference in a dynamic training environment.

Format Reward: The model is penalized for trajectories that include invalid tool calls or if the context/iteration exceeds the maximum limitation.
Correctness Reward: For trajectories without format errors, rewards are based on the comparison between the model's answer and the ground truth.

To promote efficiency, a gamma-decay factor is applied to correct trajectories. Concretely, the reward of step $i$ becomes $r\times\gamma^{T - i}$, where $r$ is the outcome reward, $T$ is the number of steps, and $0<\gamma<1$ represents the gamma-decay coeficient. This encourages the model to discover shorter, more efficient exploration. For example, while two correct trajectories may receive equal final rewards, the shorter one earns a higher reward for its initial actions.

Context management

A long-horizon research trajectory may involve massive observation contexts, and a naive agent without memory management can easily exceed the limitation within 10 iterations. To address this, we design a context-management mechanism that allows the model to retain important information while discarding unnecessary documents, thereby extending a single rollout trajectory to over 50 iterations. An early ablation study shows that a model trained with context management uses 30% more iterations, which enables it to acquire more information and achieve higher performance.

Large-scale agent RL infra

Figure 5

To address the efficiency and stability challenges of large-scale Agent RL, we have developed a suite of infrastructure with the following key features:

Fully asynchronous rollout: We implement a fully asynchronous rollout system with extensible Gym-like interfaces. The server-based architecture efficiently orchestrates actor rollouts, environmental interactions, and reward calculations in parallel. This design significantly outperforms its synchronous counterpart by eliminating resource idle time.
Turn-level partial rollout: During Agent RL, while the majority of tasks completed at the early stage, a small fraction required extensive turns. To solve this long-tail problem, we designed a Turn-level Partial Rollout mechanism. Concretely, tasks that exceed a time budget would be saved to a replay buffer. In subsequent iterations, the remaining turns would be executed with updated model weights. Combined with adapted algorithms, this mechanism delivers substantial rollout acceleration (at least 1.5x).
Robust sandbox environment: Our unified sandbox architecture eliminates inter-container overhead while maintaining isolation. Zero-downtime scheduling with Kubernetes-based hybrid cloud architecture enables dynamic resource allocation. Agent-tool communication via Model Context Protocol (MCP) maintains stateful sessions with reconnection capabilities. Our implementation supports multi-replica deployment, ensuring fault-tolerant operation and high availability in production environments.

Emerging agentic capacities

During end-to-end reinforcement learning, we observed several notable emergent abilities in Kimi-Researcher. Here are two highlights:

When presented with conflicting information from multiple sources, Kimi-Researcher resolves inconsistencies through iterative hypothesis refinement and self-correction.

"Strange Stories from a Chinese Studio" is a collection of classical Chinese short stories written by the Qing Dynasty novelist Pu Songling. The earliest manuscript copies were already in circulation during the Kangxi reign of the Qing Dynasty, and the collection comprises over four hundred short stories in total. In Volume Ten of "Strange Stories from a Chinese Studio," there is a story titled "The Green-Clothed Girl." In this story, **how many sentences did the scholar Yu Jing speak with her?**

...Now I will use the search tool. ...Numbers [^5^] and [^6^] are **vernacular translations from Himalaya**...In addition, numbers [^16^] and [^17^] are **the original texts**, I need to cross-reference them again...The dialogue between Yu Jing and the woman in green may have different records in **different versions**... ...I counted carefully, and in the vernacular translation of number [^5^], Yu Jing seems to have said **six sentences**. But in the original texts of numbers [^16^], [^44^], [^48^], etc., counting them up, Yu Jing said **four sentences**...Well, according to my previous compilation, numbers [^68^], [^88^], [^89^], [^90^], etc. all provide the original text of "The Woman in Green"...So, Yu Jing said a total of four sentences. Previously, when I counted six sentences in the translation of number [^5^], it may be because the translation converted some narrative sentences into direct quotations... ...Considering that in classical Chinese, these direct quotations are explicit, while if the translation added some dialogue, it may have been embellishment done for ease of understanding... ...After careful verification, the words Yu Jing directly said in the original text are: 1. "Your voice is delicate and soft, if you were to sing a song, it would surely be soul-stirring." 2. "Why are you so deeply suspicious and fearful?" 3. "A stirring heart and wandering eyes are common occurrences, why do you say such things?" 4. "Yes." Therefore, Yu Jing spoke a total of four sentences in the story. **Previously, in the translation of number [^5^], I mistakenly took some narrative sentences as dialogue, leading to a miscount. After confirmation, the original classical Chinese text is authoritative and has been verified multiple times.** It seems that in "The Woman in Green," Yu Jing indeed said four sentences. I think I can answer this question now. Answer: **4** In "The Woman in Green" from Volume 10 of "Strange Tales from a Chinese Studio," the scholar Yu Jing spoke a total of **4** sentences.

Kimi-Researcher demonstrates caution and rigor: even for seemingly straightforward questions, it deliberately performs additional searches and cross-validates information before answering.

A country located in South America, where the most popular sport is baseball, not football. Football is relatively less popular in this country, which has never qualified for the FIFA World Cup finals but has hosted the Copa América. In terms of football, the country has had a star player who once played in the Bundesliga. Who might that be?

...From the previous search results, I can infer: This South American country is **Venezuela**. Its most popular sport is baseball. It has never entered the World Cup finals. It hosted the Copa América (2007). Football star **Juan Arango** once played for Borussia Mönchengladbach in the Bundesliga. To confirm whether Juan Arango is a Venezuelan football star who once played in the Bundesliga, I... ...To further confirm, I want to search for more information about Juan Arango to verify... ...I previously assumed it was Venezuela, but need further verification... ...To further confirm the accuracy of the information...need to re-verify. ...Additionally, I also need to search for other players who might fit, perhaps whether Venezuelan players like Caracas, Torres, Maldonado, etc. have ever played in the Bundesliga...covering information in both Chinese and English to improve the comprehensiveness of the search. ...According to the information from references [^62^] and [^66^]...this information all supports that **Juan Fernando Arango** is Venezuela's star player in football who once played in the Bundesliga. Combined with the previous search results... To obtain more official and authoritative information for further confirmation, I decided to call up the complete content of reference [^62^]...I now feel that I have collected enough information to answer this question completely. Answer: **Juan Fernando Arango**

Use cases

What's next

Kimi-Researcher is beginning its gradual rollout to users today. It empowers you to conduct deep, comprehensive research on any topic directly within Kimi. Join the waitlist here.

It represents the early stage of our broader vision: evolving from a focused search and reasoning agent into a general-purpose agent capable of solving a wide range of complex tasks with an ever-expanding toolkit. To realize this vision, we are expanding the agent's capabilities across both tools and task domains, while also advancing the underlying reinforcement learning infrastructure and algorithms to ensure greater training stability and efficiency.

To facilitate more research efforts in the field, we are planning on open-sourcing the base pretrained model as well as the reinforcement-learned model underlying Kimi-Researcher in the following months.

Kimi-Dev

2025-06-17T00:00:00+00:00

We introduce Kimi-Dev-72B, our new open-source coding LLM for software engineering tasks. Kimi-Dev-72B achieves a new state-of-the-art on SWE-bench Verified among open-source models.

Kimi-Dev-72B achieves 60.4% performance on SWE-bench Verified. It surpasses the runner-up, setting a new state-of-the-art result among open-source models.
Kimi-Dev-72B is optimized via large-scale reinforcement learning. It autonomously patches real repositories in Docker and gains rewards only when the entire test suite passes. This ensures correct and robust solutions, aligning with real-world development standards.
Kimi-Dev-72B is available for download and deployment on Hugging Face and GitHub. We welcome developers and researchers to explore its capabilities and contribute to development.

Performance of Open-source Models on SWE-bench Verified.

⚙️ Installation

# clone repo
git clone https://github.com/MoonshotAI/Kimi-Dev.git
# create env
conda create -n kimidev python=3.12
# local install
pip install -e .

🛠️ How to use

Prepare repo structure [From Agentless]

Since for each issue in the benchmark (both SWE-Bench Lite and SWE-Bench Verified) we need to checkout the repository and process the files, you might want to save some time by downloading the preprocessed data here: swebench_repo_structure.zip. After downloading, please unzip and export the location as such

export PROJECT_FILE_LOC={folder which you saved}

Deploy vLLM Model

Installation

# Install vLLM with CUDA 12.8.
# If you are using pip.
pip install vllm --extra-index-url https://download.pytorch.org/whl/cu128
# If you are using uv.
uv pip install vllm --torch-backend=auto

Serving

vllm serve Kimi-Dev-72B --served-model-name kimi-dev --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.95 --max-seq-len-to-capture 131072 --tensor-parallel-size 8

Rollout

Kimi-Dev adopts a simplified two-stage framework for handling code repair and test writing tasks:

File Localization: Intelligently identify key files that need modification based on problem descriptions and repository structure
Code Editing: Perform precise code modifications on the located files, including bug fixes or unit test insertions

Compared to multi-step localization methods, we perform localization at the file level and then pass the complete file to the repair step for more detailed reasoning.

Run rollout script:

conda activate kimidev
# Bugfixer
python kimidev/examples/rollout_messages_bugfixer.py --model_name {vllm_serve_model}
# Testwriter
python kimidev/examples/rollout_messages_testwriter.py --model_name {vllm_serve_model}

👀 Example Results

We provide some example result files as well as the files required for test-time scaling here.

You can also download these files from Google Drive.

💪 Contributing

Welcome to submit Pull Requests or create Issues to help improve the project.

😺 Contact

If you have any questions, please feel free to submit a GitHub issue or contact zhuhan@moonshot.cn.

📝 Citation

If you find our code and models useful, please kindly cite the following information.

@misc{yang2025kimidevagentlesstrainingskill,
      title={Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents}, 
      author={Zonghan Yang and Shengjie Wang and Kelin Fu and Wenyang He and Weimin Xiong and Yibo Liu and Yibo Miao and Bofei Gao and Yejie Wang and Yingwei Ma and Yanhao Li and Yue Liu and Zhenxing Hu and Kaitai Zhang and Shuyi Wang and Huarong Chen and Flood Sung and Yang Liu and Yang Gao and Zhilin Yang and Tianyu Liu},
      year={2025},
      eprint={2509.23045},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2509.23045}, 
}

Kimi-Audio

2025-04-26T00:00:00+00:00

Kimi-Audio-7B 🤗 | Kimi-Audio-7B-Instruct 🤗 | 📑 Paper

We present Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation. This repository contains the official implementation, models, and evaluation toolkit for Kimi-Audio.

🔥🔥🔥 News!!

May 29, 2025: 👋 We release a finetuning example of Kimi-Audio-7B.
April 27, 2025: 👋 We release pretrained model weights of Kimi-Audio-7B.
April 25, 2025: 👋 We release the inference code and model weights of Kimi-Audio-7B-Instruct.
April 25, 2025: 👋 We release the audio evaluation toolkit Kimi-Audio-Evalkit. We can easily reproduce the our results and baselines by this toolkit!
April 25, 2025: 👋 We release the technical report of Kimi-Audio.

Introduction

Kimi-Audio is designed as a universal audio foundation model capable of handling a wide variety of audio processing tasks within a single unified framework. Key features include:

Universal Capabilities: Handle diverse tasks like automatic speech recognition (ASR), audio question answering (AQA), automatic audio captioning (AAC), speech emotion recognition (SER), sound event/scene classification (SEC/ASC), and end-to-end speech conversation.
State-of-the-Art Performance: Achieve SOTA results on numerous audio benchmarks (see Evaluation and the Technical Report).
Large-Scale Pre-training: Pre-train on over 13 million hours of diverse audio data (speech, music, sounds) and text data, enabling robust audio reasoning and language understanding.
Novel Architecture: Employ a hybrid audio input (continuous acoustic vectors + discrete semantic tokens) and an LLM core with parallel heads for text and audio token generation.
Efficient Inference: Feature a chunk-wise streaming detokenizer based on flow matching for low-latency audio generation.
Open-Source: Release the code and model checkpoints for both pre-training and instruction fine-tuning, and release a comprehensive evaluation toolkit to foster community research and development.

Architecture Overview

Kimi-Audio consists of three main components:

Audio Tokenizer: Converts input audio into:
- Discrete semantic tokens (12.5Hz) using vector quantization.
- Continuous acoustic features derived from a Whisper encoder (downsampled to 12.5Hz).
Audio LLM: A transformer-based model (initialized from a pre-trained text LLM like Qwen 2.5 7B) with shared layers processing multimodal inputs, followed by parallel heads for autoregressively generating text tokens and discrete audio semantic tokens.
Audio Detokenizer: Converts the predicted discrete semantic audio tokens back into high-fidelity waveforms using a flow-matching model and a vocoder (BigVGAN), supporting chunk-wise streaming with a look-ahead mechanism for low latency.

Getting Started

Step1: Get the Code

git clone https://github.com/MoonshotAI/Kimi-Audio.git
cd Kimi-Audio
git submodule update --init --recursive
pip install -r requirements.txt

Kimi‑Audio can now be installed directly via pip.

pip install torch
pip install git+https://github.com/MoonshotAI/Kimi-Audio.git

Quick Start

This example demonstrates basic usage for generating text from audio (ASR) and generating both text and speech in a conversational turn.

import soundfile as sf
from kimia_infer.api.kimia import KimiAudio

# --- 1. Load Model ---
model_path = "moonshotai/Kimi-Audio-7B-Instruct" 
model = KimiAudio(model_path=model_path, load_detokenizer=True)

# --- 2. Define Sampling Parameters ---
sampling_params = {
    "audio_temperature": 0.8,
    "audio_top_k": 10,
    "text_temperature": 0.0,
    "text_top_k": 5,
    "audio_repetition_penalty": 1.0,
    "audio_repetition_window_size": 64,
    "text_repetition_penalty": 1.0,
    "text_repetition_window_size": 16,
}

# --- 3. Example 1: Audio-to-Text (ASR) ---
messages_asr = [
    # You can provide context or instructions as text
    {"role": "user", "message_type": "text", "content": "Please transcribe the following audio:"},
    # Provide the audio file path
    {"role": "user", "message_type": "audio", "content": "test_audios/asr_example.wav"}
]

# Generate only text output
_, text_output = model.generate(messages_asr, **sampling_params, output_type="text")
print(">>> ASR Output Text: ", text_output) # Expected output: "这并不是告别，这是一个篇章的结束，也是新篇章的开始。"


# --- 4. Example 2: Audio-to-Audio/Text Conversation ---
messages_conversation = [
    # Start conversation with an audio query
    {"role": "user", "message_type": "audio", "content": "test_audios/qa_example.wav"}
]

# Generate both audio and text output
wav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type="both")

# Save the generated audio
output_audio_path = "output_audio.wav"
sf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output
print(f">>> Conversational Output Audio saved to: {output_audio_path}")
print(">>> Conversational Output Text: ", text_output) # Expected output: "当然可以，这很简单。一二三四五六七八九十。"

# --- 5. Example 3: Audio-to-Audio/Text Conversation with Multiturn ---

messages = [
    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q1.wav"},
    # This is the first turn output of Kimi-Audio
    {"role": "assistant", "message_type": "audio-text", "content": ["test_audios/multiturn/case2/multiturn_a1.wav", "当然可以，这很简单。一二三四五六七八九十。"]},
    {"role": "user", "message_type": "audio", "content": "test_audios/multiturn/case2/multiturn_q2.wav"}
]
wav, text = model.generate(messages, **sampling_params, output_type="both")


# Generate both audio and text output
wav_output, text_output = model.generate(messages_conversation, **sampling_params, output_type="both")

# Save the generated audio
output_audio_path = "output_audio.wav"
sf.write(output_audio_path, wav_output.detach().cpu().view(-1).numpy(), 24000) # Assuming 24kHz output
print(f">>> Conversational Output Audio saved to: {output_audio_path}")
print(">>> Conversational Output Text: ", text_output) # Expected output: "没问题，继续数下去就是十一十二十三十四十五十六十七十八十九二十。"

print("Kimi-Audio inference examples complete.")

Evaluation

Kimi-Audio achieves state-of-the-art (SOTA) performance across a wide range of audio benchmarks.

The below is the overall performance:

Here are performances on different benchmarks, you can easily reproduce the our results and baselines by our Kimi-Audio-Evalkit (also see Evaluation Toolkit):

Automatic Speech Recognition (ASR)

Datasets	Model	Performance (WER↓)
LibriSpeech test-clean \| test-other	Qwen2-Audio-base	1.74 \| 4.04
Baichuan-base	3.02 \| 6.04
Step-Audio-chat	3.19 \| 10.67
Qwen2.5-Omni	2.37 \| 4.21
Kimi-Audio	1.28 \| 2.42
Fleurs zh \| en	Qwen2-Audio-base	3.63 \| 5.20
Baichuan-base	4.15 \| 8.07
Step-Audio-chat	4.26 \| 8.56
Qwen2.5-Omni	2.92 \| 4.17
Kimi-Audio	2.69 \| 4.44
AISHELL-1	Qwen2-Audio-base	1.52
Baichuan-base	1.93
Step-Audio-chat	2.14
Qwen2.5-Omni	1.13
Kimi-Audio	0.60
AISHELL-2 ios	Qwen2-Audio-base	3.08
Baichuan-base	3.87
Step-Audio-chat	3.89
Qwen2.5-Omni	2.56
Kimi-Audio	2.56
WenetSpeech test-meeting \| test-net	Qwen2-Audio-base	8.40 \| 7.64
Baichuan-base	13.28 \| 10.13
Step-Audio-chat	10.83 \| 9.47
Qwen2.5-Omni	7.71 \| 6.04
Kimi-Audio	6.28 \| 5.37
Kimi-ASR Internal Testset subset1 \| subset2	Qwen2-Audio-base	2.31 \| 3.24
Baichuan-base	3.41 \| 5.60
Step-Audio-chat	2.82 \| 4.74
Qwen2.5-Omni	1.53 \| 2.68
Kimi-Audio	1.42 \| 2.44

Audio Understanding

Datasets	Model	Performance↑
MMAU music \| sound \| speech	Qwen2-Audio-base	58.98 \| 69.07 \| 52.55
Baichuan-chat	49.10 \| 59.46 \| 42.47
GLM-4-Voice	38.92 \| 43.54 \| 32.43
Step-Audio-chat	49.40 \| 53.75 \| 47.75
Qwen2.5-Omni	62.16 \| 67.57 \| 53.92
Kimi-Audio	61.68 \| 73.27 \| 60.66
ClothoAQA test \| dev	Qwen2-Audio-base	71.73 \| 72.63
Baichuan-chat	48.02 \| 48.16
Step-Audio-chat	45.84 \| 44.98
Qwen2.5-Omni	72.86 \| 73.12
Kimi-Audio	71.24 \| 73.18
VocalSound	Qwen2-Audio-base	93.82
Baichuan-base	58.17
Step-Audio-chat	28.58
Qwen2.5-Omni	93.73
Kimi-Audio	94.85
Nonspeech7k	Qwen2-Audio-base	87.17
Baichuan-chat	59.03
Step-Audio-chat	21.38
Qwen2.5-Omni	69.89
Kimi-Audio	93.93
MELD	Qwen2-Audio-base	51.23
Baichuan-chat	23.59
Step-Audio-chat	33.54
Qwen2.5-Omni	49.83
Kimi-Audio	59.13
TUT2017	Qwen2-Audio-base	33.83
Baichuan-base	27.9
Step-Audio-chat	7.41
Qwen2.5-Omni	43.27
Kimi-Audio	65.25
CochlScene test \| dev	Qwen2-Audio-base	52.69 \| 50.96
Baichuan-base	34.93 \| 34.56
Step-Audio-chat	10.06 \| 10.42
Qwen2.5-Omni	63.82 \| 63.82
Kimi-Audio	79.84 \| 80.99

Audio-to-Text Chat

Datasets	Model	Performance↑
OpenAudioBench AlpacaEval \| Llama Questions \| Reasoning QA \| TriviaQA \| Web Questions	Qwen2-Audio-chat	57.19 \| 69.67 \| 42.77 \| 40.30 \| 45.20
Baichuan-chat	59.65 \| 74.33 \| 46.73 \| 55.40 \| 58.70
GLM-4-Voice	57.89 \| 76.00 \| 47.43 \| 51.80 \| 55.40
StepAudio-chat	56.53 \| 72.33 \| 60.00 \| 56.80 \| 73.00
Qwen2.5-Omni	72.76 \| 75.33 \| 63.76 \| 57.06 \| 62.80
Kimi-Audio	75.73 \| 79.33 \| 58.02 \| 62.10 \| 70.20
VoiceBench AlpacaEval \| CommonEval \| SD-QA \| MMSU	Qwen2-Audio-chat	3.69 \| 3.40 \| 35.35 \| 35.43
Baichuan-chat	4.00 \| 3.39 \| 49.64 \| 48.80
GLM-4-Voice	4.06 \| 3.48 \| 43.31 \| 40.11
StepAudio-chat	3.99 \| 2.99 \| 46.84 \| 28.72
Qwen2.5-Omni	4.33 \| 3.84 \| 57.41 \| 56.38
Kimi-Audio	4.46 \| 3.97 \| 63.12 \| 62.17
VoiceBench OpenBookQA \| IFEval \| AdvBench \| Avg	Qwen2-Audio-chat	49.01 \| 22.57 \| 98.85 \| 54.72
Baichuan-chat	63.30 \| 41.32 \| 86.73 \| 62.51
GLM-4-Voice	52.97 \| 24.91 \| 88.08 \| 57.17
StepAudio-chat	31.87 \| 29.19 \| 65.77 \| 48.86
Qwen2.5-Omni	79.12 \| 53.88 \| 99.62 \| 72.83
Kimi-Audio	83.52 \| 61.10 \| 100.00 \| 76.93

Speech Conversation

Performance of Kimi-Audio and baseline models on speech conversation.

Model	Ability
GPT-4o	4.21	3.65	4.05	3.87	4.54	4.06
Step-Audio-chat	3.25	2.87	3.33	3.05	4.14	3.33
GLM-4-Voice	3.83	3.51	3.77	3.07	4.04	3.65
GPT-4o-mini	3.15	2.71	4.24	3.16	4.01	3.45
Kimi-Audio	4.30	3.45	4.27	3.39	4.09	3.90

Finetune

We release the pre-trained model and the lightweight finetune codes. Please refer to the finetune_codes/README.md for more details.

Evaluation Toolkit

Evaluating and comparing audio foundation models is challenging due to inconsistent metrics, varying inference configurations, and a lack of standardized generation evaluation. To address this, we developed and open-sourced an Evaluation Toolkit.

Key features:

Integrates Kimi-Audio and other recent audio LLMs.
Implements standardized metric calculation and integrates LLMs for intelligent judging (e.g., for AQA).
Provides a unified platform for side-by-side comparisons with shareable inference 'recipes' for reproducibility.
Includes a benchmark for evaluating speech conversation abilities (control, empathy, style).

We encourage the community to use and contribute to this toolkit to foster more reliable and comparable benchmarking. Find it here: Kimi-Audio-Evalkit.

Generation Testset

We collect and release Kimi-Audio-Generation-Testset, which is designed to benchmark and evaluate the conversational capabilities of audio-based dialogue models. It consists of a collection of audio files containing various instructions and conversational prompts. The primary goal is to assess a model's ability to generate not just relevant, but also appropriately styled audio responses. The language in dataset is Chinese.

License

The model is based and modified from Qwen 2.5-7B. Code derived from Qwen2.5-7B is licensed under the Apache 2.0 License. Other parts of the code are licensed under the MIT License.

Acknowledgements

We would like to thank the following projects and individuals for their contributions to the development of Kimi-Audio:

Thank you to all the open-source projects for their contributions to this project!

Citation

If you find Kimi-Audio useful in your research or applications, please cite our technical report:

@misc{kimiteam2025kimiaudiotechnicalreport,
      title={Kimi-Audio Technical Report}, 
      author={KimiTeam and Ding Ding and Zeqian Ju and Yichong Leng and Songxiang Liu and Tong Liu and Zeyu Shang and Kai Shen and Wei Song and Xu Tan and Heyi Tang and Zhengtao Wang and Chu Wei and Yifei Xin and Xinran Xu and Jianwei Yu and Yutao Zhang and Xinyu Zhou and Y. Charles and Jun Chen and Yanru Chen and Yulun Du and Weiran He and Zhenxing Hu and Guokun Lai and Qingcheng Li and Yangyang Liu and Weidong Sun and Jianzhou Wang and Yuzhi Wang and Yuefeng Wu and Yuxin Wu and Dongchao Yang and Hao Yang and Ying Yang and Zhilin Yang and Aoxiong Yin and Ruibin Yuan and Yutong Zhang and Zaida Zhou},
      year={2025},
      eprint={2504.18425},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2504.18425}, 
}

Contact Us

For questions, issues, or collaboration inquiries, please feel free to open an issue on GitHub.

Kimina-Prover Preview

2025-04-15T00:00:00+00:00

📈 Introducing Kimina-Prover Preview, the first large formal reasoning model that can reason in a human-like way and prove mathematical theorems rigorously in the Lean 4 language.

Some key ingredients about the design and training of Kimina-Prover Preview are listed as follows.

@article{kimina_prover_2025,
	title = {Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning},
	author = {Wang, Haiming and Unsal, Mert and Lin, Xiaohan and Baksys, Mantas and Liu, Junqi and Santos, Marco Dos and Sung, Flood and Vinyes, Marina and Ying, Zhenzhe and Zhu, Zekai and Lu, Jianqiao and Saxcé, Hugues de and Bailey, Bolton and Song, Chendong and Xiao, Chenjun and Zhang, Dehao and Zhang, Ebony and Pu, Frederick and Zhu, Han and Liu, Jiawei and Bayer, Jonas and Michel, Julien and Yu, Longhui and Dreyfus-Schmidt, Léo and Tunstall, Lewis and Pagani, Luigi and Machado, Moreira and Bourigault, Pauline and Wang, Ran and Polu, Stanislas and Barroyer, Thibaut and Li, Wen-Ding and Niu, Yazhe and Fleureau, Yann and Hu, Yangyang and Yu, Zhouliang and Wang, Zihan and Yang, Zhilin and Liu, Zhengying and Li, Jia},
	year = {2025},
	url = {http://arxiv.org/abs/2504.11354},
}

Kimi-VL

2025-04-10T00:00:00+00:00

1. Introduction

We present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities—all while activating only 2.8B parameters in its language decoder (Kimi-VL-A3B).

Kimi-VL demonstrates strong performance across challenging domains: as a general-purpose VLM, Kimi-VL excels in multi-turn agent interaction tasks (e.g.,OSWorld), achieving state-of-the-art results comparable to flagship models. Furthermore, it exhibits remarkable capabilities across diverse challenging vision language tasks, including college-level image and video comprehension, optical character recognition (OCR), mathematical reasoning, multi-image understanding, and etc.

In comparative evaluations, it effectively competes with cutting-edge efficient VLMs such as GPT-4o-mini, Qwen2.5-VL-7B, and Gemma-3-12B-IT, while surpassing GPT-4o in several specialized domains.

Kimi-VL also advances the pareto frontiers of multimodal models in processing long contexts and perceiving clearly: Equipped with a 128K extended context window, Kimi-VL can processes long and diverse inputs, achieving impressive scores of 64.5 on LongVideoBench, and 35.1 on MMLongBench-Doc; Its native-resolution vision encoder, MoonViT, further allows it to see and understand ultra-high-resolution visual inputs, achieving 83.2 on InfoVQA and 34.5 on ScreenSpot-Pro, while maintaining lower computational cost with common visual inputs and general tasks.

Building on this foundation, we introduce an advanced long-thinking variant: Kimi-VL-Thinking. Developed through long chain-of-thought (CoT) supervised fine-tuning (SFT) and reinforcement learning (RL), this model exhibits strong long-horizon reasoning capabilities. It achieves scores of 61.7 on MMMU, 36.8 on MathVision, and 71.3 on MathVista while maintaining the compact 2.8B activated LLM parameter footprint, setting a new standard for efficient yet capable multimodal thinking models.

Besides original model variants, we also provide a new Kimi-VL-A3B-Thinking-2506 variant with several new or improved abilities:

It Thinks Smarter while Consuming Less Tokens: The 2506 version reaches better accuracy on multimodal reasoning benchmarks: 56.9 on MathVision (+20.1), 80.1 on MathVista (+8.4), 46.3 on MMMU-Pro (+3.2), 64.0 on MMMU (+2.1), while in average reducing 20% thinking length.
It Sees Clearer with Thinking: Unlike the previous version that specializes on thinking tasks, the 2506 version can also achieve the same or even better ability on general visual perception and understanding, e.g. MMBench-EN-v1.1 (84.4), MMStar (70.4), RealWorldQA (70.0), MMVet (78.4) compared to the original non-thinking version (Kimi-VL-A3B-Instruct).
It Extends to Video Scenarios: The new 2506 version also improves on video reasoning and understanding benchmarks. It sets new state-of-the-art for open-source models on VideoMMMU (65.2), while also retaining good ability on general video understanding (71.9 on Video-MME).
It Extends to Higher Resolution: The new 2506 version supports 3.2 million total pixels in a single image (1792x1792), 4X compared to the original release. This leads to non-trivial improvements on high-resolution perception and OS-agent grounding benchmarks: 83.2 on V* Benchmark (without extra tools), 52.8 on ScreenSpot-Pro, 52.5 on OSWorld-G (full set with refusal).

2. Architecture

The model adopts an MoE language model, a native-resolution visual encoder (MoonViT), and an MLP projector, as illustrated in the following image.

3. News

2025.06.21: Release of Kimi-VL-A3B-Thinking-2506: Tech Blog & Cookbook, 🤗 Hugging Face
2025.04.15: vLLM has supported Kimi-VL deployment. See #16387 for details.
2025.04.14: LLaMA-Factory has supported Kimi-VL finetuning. See #7719 for details.

4. Model Variants

🤗 For common general multimodal perception and understanding, OCR, long video and long document, video perception, and OS-agent uses, we recommend Kimi-VL-A3B-Instruct for efficient inference; meanwhile, our new thinking version, Kimi-VL-A3B-Thinking-2506 also has excellent multimodal perception, long video and long document and OS-agent grounding abilities while achieving better multimodal reasoning skills. See this blog for more information.

Model	#Total Params	#Activated Params	Context Length	Download Link
🔥Kimi-VL-A3B-Thinking-2506	16B	3B	128K	🤗 Hugging Face
Kimi-VL-A3B-Instruct	16B	3B	128K	🤗 Hugging Face
Kimi-VL-A3B-Thinking (deprecated)	16B	3B	128K	🤗 Hugging Face

Note

Recommended parameter settings:

For Thinking models, it is recommended to use Temperature = 0.8.
For Instruct models, it is recommended to use Temperature = 0.2.

Hugging Face Demo

🤗 We serve our model demo in Hugging Face spaces:

Chat with Kimi-VL-A3B-Thinking-2506👀🤔🗺️🎬📖🖥️ (unifying thinking, general understanding, puzzle solving, agent, video, PDF) model on Chat Web.

5. Performance

Note

See the performance of Kimi-VL-A3B-Thinking-2506 at Hugging Face.

As an efficient model, Kimi-VL can robustly handle diverse tasks (fine-grained perception, math, college-level problems, OCR, agent, etc) across a broad spectrum of input forms (single-image, multi-image, video, long-document, etc).

A brief comparison with existing 10B-level dense VLMs and DeepSeek-VL2 (A4.5B):

With effective long-thinking abilities, Kimi-VL-A3B-Thinking (2504 version) can match the performance of 30B/70B frontier open-source VLMs on MathVision benchmark:

6. Example usage

Setup

conda create -n kimi-vl python=3.10 -y
conda activate kimi-vl
pip install -r requirements.txt

Note

If you encounter Out-of-Memory or want to speed up inference, please install flash-attn with pip install flash-attn --no-build-isolation.

Inference with Hugging Face Transformers

We introduce how to use our model at inference stage using transformers library. It is recommended to use python=3.10, torch=2.5.1, and transformers=4.51.3 as the development environment.

Kimi-VL-A3B-Instruct:

import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoProcessor

model_path = "moonshotai/Kimi-VL-A3B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
# If flash-attn has been installed, it is recommended to set torch_dtype=torch.bfloat16 and attn_implementation="flash_attention_2"
# to save memory and speed up inference
# model = AutoModelForCausalLM.from_pretrained(
#     model_path,
#     torch_dtype=torch.bfloat16,
#     device_map="auto",
#     trust_remote_code=True,
#     attn_implementation="flash_attention_2"
# )

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

image_path = "./figures/demo.png"
image = Image.open(image_path)
messages = [
    {"role": "user", "content": [{"type": "image", "image": image_path}, {"type": "text", "text": "What is the dome building in the picture? Think step by step."}]}
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
inputs = processor(images=image, text=text, return_tensors="pt", padding=True, truncation=True).to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=512)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
response = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]
print(response)

Kimi-VL-A3B-Thinking-2506:

import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoProcessor

model_path = "moonshotai/Kimi-VL-A3B-Thinking-2506"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
# If flash-attn has been installed, it is recommended to set torch_dtype=torch.bfloat16 and attn_implementation="flash_attention_2"
# to save memory and speed up inference
# model = AutoModelForCausalLM.from_pretrained(
#     model_path,
#     torch_dtype=torch.bfloat16,
#     device_map="auto",
#     trust_remote_code=True,
#     attn_implementation="flash_attention_2"
# )
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

image_paths = ["./figures/demo1.png", "./figures/demo2.png"]
images = [Image.open(path) for path in image_paths]
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image_path} for image_path in image_paths
        ] + [{"type": "text", "text": "Please infer step by step who this manuscript belongs to and what it records"}],
    },
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
inputs = processor(images=images, text=text, return_tensors="pt", padding=True, truncation=True).to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=32768, temperature=0.8)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
response = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]
print(response)

7. Finetuning

Collaborating closely with the open-source community, Kimi-VL now offers seamless support for efficient fine-tuning through the latest version of LLaMA-Factory.

The framework enables Single-GPU LoRA fine-tuning with 50GB of VRAM, as well as Multi-GPU full/lora fine-tuning using DeepSpeed ZeRO-2. For more detailed configuration instructions, check out this PR.

8. Deployment

Using vLLM

The vLLM main branch has supported Kimi-VL deployment. You are welcome to deploy Kimi-VL using vLLM.

Offline Inference

from PIL import Image
from transformers import AutoProcessor
from vllm import LLM, SamplingParams

model_path = "moonshotai/Kimi-VL-A3B-Instruct"  # or "moonshotai/Kimi-VL-A3B-Thinking-2506"
llm = LLM(
    model_path,
    trust_remote_code=True,
)

processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

image_path = "./figures/demo.png"
image = Image.open(image_path)
messages = [
    {"role": "user", "content": [{"type": "image", "image": image_path}, {"type": "text", "text": "What is the dome building in the picture? Think step by step."}]}
]
text = processor.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = llm.generate([{"prompt": text, "multi_modal_data": {"image": image}}], sampling_params = SamplingParams(max_tokens=512))

print("-" * 50)
for o in outputs:
    generated_text = o.outputs[0].text
    print(generated_text)
    print("-" * 50)

OpenAI-Compatible Server

Serve Kimi-VL with vllm serve command:

# If you need a longer context window, you can set --max-model-len and --max-num-batched-tokens to 131072
# If you need more input images, you can set --limit-mm-per-prompt image=256 or 512

# kimi-vl-thinking-2506
vllm serve moonshotai/Kimi-VL-A3B-Thinking-2506 --served-model-name kimi-vl-thinking-2506 --trust-remote-code --tensor-parallel-size 1 --max-num-batched-tokens 32768 --max-model-len 32768 --limit-mm-per-prompt image=64

# kimi-vl-instruct
vllm serve moonshotai/Kimi-VL-A3B-Instruct --served-model-name kimi-vl --trust-remote-code --tensor-parallel-size 1 --max-num-batched-tokens 32768 --max-model-len 32768 --limit-mm-per-prompt image=64

Call the API

import base64
from PIL import Image
from io import BytesIO
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="token-abc123",
)

image_path = "./figures/demo.png"
image = Image.open(image_path).convert("RGB")

buffered = BytesIO()
image.save(buffered, format="JPEG")
img_b64_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
base64_image_url = f"data:image/jpeg;base64,{img_b64_str}"

messages = [
    {"role": "user", "content": [{"type": "image_url", "image_url": {"url": base64_image_url}}, {"type": "text", "text": "What is the dome building in the picture? Think step by step."}]}
]

completion = client.chat.completions.create(
  model="kimi-vl-thinking-2506", # or kimi-vl
  messages=messages
)

print(completion.choices[0].message)

9. Citation

@misc{kimiteam2025kimivltechnicalreport,
      title={{Kimi-VL} Technical Report}, 
      author={Kimi Team and Angang Du and Bohong Yin and Bowei Xing and Bowen Qu and Bowen Wang and Cheng Chen and Chenlin Zhang and Chenzhuang Du and Chu Wei and Congcong Wang and Dehao Zhang and Dikang Du and Dongliang Wang and Enming Yuan and Enzhe Lu and Fang Li and Flood Sung and Guangda Wei and Guokun Lai and Han Zhu and Hao Ding and Hao Hu and Hao Yang and Hao Zhang and Haoning Wu and Haotian Yao and Haoyu Lu and Heng Wang and Hongcheng Gao and Huabin Zheng and Jiaming Li and Jianlin Su and Jianzhou Wang and Jiaqi Deng and Jiezhong Qiu and Jin Xie and Jinhong Wang and Jingyuan Liu and Junjie Yan and Kun Ouyang and Liang Chen and Lin Sui and Longhui Yu and Mengfan Dong and Mengnan Dong and Nuo Xu and Pengyu Cheng and Qizheng Gu and Runjie Zhou and Shaowei Liu and Sihan Cao and Tao Yu and Tianhui Song and Tongtong Bai and Wei Song and Weiran He and Weixiao Huang and Weixin Xu and Xiaokun Yuan and Xingcheng Yao and Xingzhe Wu and Xinxing Zu and Xinyu Zhou and Xinyuan Wang and Y. Charles and Yan Zhong and Yang Li and Yangyang Hu and Yanru Chen and Yejie Wang and Yibo Liu and Yibo Miao and Yidao Qin and Yimin Chen and Yiping Bao and Yiqin Wang and Yongsheng Kang and Yuanxin Liu and Yulun Du and Yuxin Wu and Yuzhi Wang and Yuzi Yan and Zaida Zhou and Zhaowei Li and Zhejun Jiang and Zheng Zhang and Zhilin Yang and Zhiqi Huang and Zihao Huang and Zijia Zhao and Ziwei Chen},
      year={2025},
      eprint={2504.07491},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.07491}, 
}

Muon is Scalable for LLM Training

2025-02-23T00:00:00+00:00

Abstract

Recently, the Muon optimizer based on matrix orthogonalization has demonstrated strong results in training small-scale language models, but the scalability to larger models has not been proven. We identify two crucial techniques for scaling up Muon: (1) adding weight decay and (2) carefully adjusting the per-parameter update scale. These techniques allow Muon to work out-of-the-box on large-scale training without the need of hyper-parameter tuning. Scaling law experiments indicate that Muon achieves ∼ 2× computational efficiency compared to AdamW with compute optimal training.

Based on these improvements, we introduce Moonlight, a 3B/16B-parameter Mixture-of-Expert (MoE) model trained with 5.7T tokens using Muon. Our model improves the current Pareto frontier, achieving better performance with much fewer training FLOPs compared to prior models.

We open-source our distributed Muon implementation that is memory optimal and communication efficient. We also release the pretrained, instruction-tuned, and intermediate checkpoints to support future research.

Our code is available at MoonshotAI/Moonlight.

Key Ingredients

Our work builds upon Muon while systematically identifying and resolving its limitations in large-scale training scenarios. Our technical contributions include:

Analysis for Effective Scaling of Muon: Through extensive analysis, we identify that weight decay plays a crucial roles in Muon's scalability. Besides, we proposed to keep a consistent update root mean square (RMS) across different matrix and non-matrix parameters through parameter-wise update scale adjustments. Such adjustments significantly enhanced training stability.
Efficient Distributed Implementation: We develop a distributed version of Muon with ZeRO-1 style optimization, achieving optimal memory efficiency and reduced communication overhead while preserving the mathematical properties of the algorithm.
Scaling Law Validation: We performed scaling law research that compares Muon with strong AdamW baselines, and showed the superior performance of Muon (see Figure 1). Based on the scaling law results, Muon achieves comparable performance to AdamW trained counterparts while requiring only approximately 52% of the training FLOPs.

Scaling up with Muon. (a) Scaling law experiments comparing Muon and Adam. Muon is 2 times more sample efficient than Adam. (b) The MMLU performance of our Moonlight model optimized with Muon and other comparable models. Moonlight advances the Pareto frontier of performance vs training FLOPs.

Performance

We named our lightweight model trained with Muon "Moonlight". We compared Moonlight with SOTA public models at similar scale:

LLAMA3-3B is a 3B-parameter dense model trained with 9T tokens
Qwen2.5-3B is a 3B-parameter dense model trained with 18T tokens
Deepseek-v2-Lite is a 2.4B/16B-parameter MOE model trained with 5.7T tokens

	Benchmark (Metric)	Llama3.2-3B	Qwen2.5-3B	DSV2-Lite	Moonlight
	Activated Param†	2.81B	2.77B	2.24B	2.24B
	Total Params†	2.81B	2.77B	15.29B	15.29B
	Training Tokens	9T	18T	5.7T	5.7T
	Optimizer	AdamW	*	AdamW	Muon
English	MMLU	54.75	65.6	58.3	70.0
	MMLU-pro	25.0	34.6	25.5	42.4
	BBH	46.8	56.3	44.1	65.2
	TriviaQA‡	59.6	51.1	65.1	66.3
Code	HumanEval	28.0	42.1	29.9	48.1
	MBPP	48.7	57.1	43.2	63.8
Math	GSM8K	34.0	79.1	41.1	77.4
	MATH	8.5	42.6	17.1	45.3
	CMath	-	80.0	58.4	81.1
Chinese	C-Eval	-	75.0	60.3	77.2
	CMMLU	-	75.0	64.3	78.2

Qwen 2 & 2.5 reports didn't disclose their optimizer information. †The reported parameter counts exclude the embedding parameters. ‡We test all listed models with the full set of TriviaQA.

Example usage

Model Download

Model	#Total Params	#Activated Params	Context Length	Download Link
Moonlight	16B	3B	8K	🤗 Hugging Face
Moonlight-Instruct	16B	3B	8K	🤗 Hugging Face

Inference with Hugging Face Transformers

We introduce how to use our model at inference stage using transformers library. It is recommended to use python=3.10, torch>=2.1.0, and transformers=4.48.2 as the development environment.

For our pretrained model (Moonlight):

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "moonshotai/Moonlight-16B-A3B"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

prompt = "1+1=2, 1+2="
inputs = tokenizer(prompt, return_tensors="pt", padding=True, truncation=True).to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=100)
response = tokenizer.batch_decode(generated_ids)[0]
print(response)

For our instruct model (Moonlight-Instruct):

from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "moonshotai/Moonlight-16B-A3B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

messages = [
    {"role": "system", "content": "You are a helpful assistant provided by Moonshot-AI."},
    {"role": "user", "content": "Is 123 a prime?"}
]
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
generated_ids = model.generate(inputs=input_ids, max_new_tokens=500)
response = tokenizer.batch_decode(generated_ids)[0]
print(response)

Moonlight has the same architecture as DeepSeek-V3, which is supported by many popular inference engines, such as VLLM and SGLang. As a result, our model can also be easily deployed using these tools.

Training

# train qwen-like dense model with muon
python3 examples/toy_train.py --model qwen --optimizer muon --dataset openwebtext-100k --hidden_size 896 --lr 1e-3

# train qwen-like dense model with adamw
python3 examples/toy_train.py --model qwen --optimizer adamw --dataset openwebtext-100k --hidden_size 896 --lr 1e-3

Intermediate Checkpoints

To support ongoing research efforts, we will soon release our intermediate checkpoints. Coming soon...

Citation

If you find Moonlight is useful or want to use in your projects, please kindly cite our paper:

@misc{liu2025muonscalablellmtraining,
      title={Muon is Scalable for LLM Training}, 
      author={Jingyuan Liu and Jianlin Su and Xingcheng Yao and Zhejun Jiang and Guokun Lai and Yulun Du and Yidao Qin and Weixin Xu and Enzhe Lu and Junjie Yan and Yanru Chen and Huabin Zheng and Yibo Liu and Shaowei Liu and Bohong Yin and Weiran He and Han Zhu and Yuzhi Wang and Jianzhou Wang and Mengnan Dong and Zheng Zhang and Yongsheng Kang and Hao Zhang and Xinran Xu and Yutao Zhang and Yuxin Wu and Xinyu Zhou and Zhilin Yang},
      year={2025},
      eprint={2502.16982},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2502.16982}, 
}

MoBA: Mixture of Block Attention for Long-Context LLMs

2025-02-18T00:00:00+00:00

Full Report

🚀 Introducing MoBA --- Mixture of Block Attention

Trainable Block Sparse Attention: The full context is divided into blocks, where each query token learns to attend to the most relevant KV blocks, enabling efficient processing of long sequences.
Parameter-less Gating Mechanism: A novel Parameter-less top-k gating mechanism is introduced to selects the most relevant blocks for each query token, ensuring that the model focuses only on the most informative blocks.
Seamlessly Transition between Full and Sparse Attention: MoBA is designed to be a flexible substitute for full attention, allowing seamless transitions between full and sparse attention modes.

Note: MoBA requires continue training of existing models to achieve its acceleration benefits. It is not a drop-in sparse attention solution that can be directly applied to pretrained models without additional training.

Abstract

Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in computational complexity inherent in traditional attention mechanisms presents a prohibitive overhead. Existing approaches either impose strongly biased structures, such as sink or window attention which are task-specific, or radically modify the attention mechanism into linear approximations, whose performance in complex reasoning tasks remains inadequately explored.

In this work, we propose a solution that adheres to the “less structure” principle, allowing the model to autonomously determine where to attend, rather than introducing predefined biases. We introduce Mixture of Block Attention (MoBA), an innovative approach that applies the principles of Mixture of Experts (MoE) to the attention mechanism. This novel architecture demonstrates superior performance on long-context tasks while offering a key advantage: the ability to seamlessly transition between full and sparse attention, enhancing efficiency without the risk of compromising performance. MoBA has already been deployed to support Kimi’s long-context requests and demonstrates significant advancements in efficient attention computation for LLMs.

Our code is available at MoonshotAI/MoBA.

Evaluation with 1M context length

Environment Setup

Note that current kernel implementations rely on flash-attn==2.6.3 and torch >= 2.1.0

conda create -n moba python=3.10
conda activate moba
pip install .

Quick Start

We provide a transformers-friendly implementation for MoBA.

Feel free to choose attention backends by --attn between moba and moba_naive.

python3 examples/llama.py --model meta-llama/Llama-3.1-8B --attn moba

Implementation Details

moba_naive: A naive implementation based on attention masks. It's designed to help understand how MoBA selects corresponding chunks. You may save and visualize the attention masks to see the block selection process.
moba_efficient: Our production-ready implementation optimized for performance. It achieves up to 40x speedup compared to moba_naive (tested with 32K sequence length, 1 attention head, MoBA Block 2048 and MoBA Topk 3). We recommend using this version for practical applications.

Unit Tests

pytest tests/test_moba_attn.py

References

Llama Implementation: huggingface/transformers
Flash Attention: Dao-AILab/flash-attention

Citation

If you find MoBA is useful or want to use in your projects, please kindly cite our paper:

@article{lu2025mobamixtureblockattention,
  author = {Enzhe Lu and Zhejun Jiang and Jingyuan Liu and Yulun Du and Tao Jiang and Chao Hong and Shaowei Liu and Weiran He and Enming Yuan and Yuzhi Wang and Zhiqi Huang and Huan Yuan and Suting Xu and Xinran Xu and Guokun Lai and Yanru Chen and Huabin Zheng and Junjie Yan and Jianlin Su and Yuxin Wu and Yutao Zhang and Zhilin Yang and Xinyu Zhou and Mingxing Zhang and Jiezhong Qiu},
  title = {MoBA: Mixture of Block Attention for Long-Context LLMs},
  journal={arXiv preprint arXiv:2502.13189},
  year={2025}
}

Kimi K1.5: Scaling Reinforcement Learning with LLMs

2025-01-20T00:00:00+00:00

There are a few key ingredients about the design and training of k1.5.

@article{team2025kimi,
  title={Kimi k1.5: Scaling reinforcement learning with llms},
  author={Team, Kimi and Du, Angang and Gao, Bofei and Xing, Bowei and Jiang, Changjiu and Chen, Cheng and Li, Cheng and Xiao, Chenjun and Du, Chenzhuang and Liao, Chonghua and others},
  journal={arXiv preprint arXiv:2501.12599},
  year={2025}
}

Kimi Research Blog | Moonshot AI

PerceptionBench

Introducing PerceptionBench

Overview

The Dataset

Distribution of Tasks per Category

Using PerceptionBench to Compare Models

Conclusion

Kimi K3

An Open 3T-Class Model

Coding

Kernel Optimization

GPU Compiler Development

Game Dev and Digital Creation

Chip Design

Coding for Research

Knowledge Work

Research with Interactive Visualization

Case 1: Interactive 42 years of AI ASIC industry research website

Case 2: Fusion Industry Research

Case 3: GWTC-5 Gravitational-wave Analysis

Widgets and Dashboard

Video Editing

Architecture and Infrastructure

Availability

Full Benchmark Table

Footnotes

Coding benchmarks

Productivity and agentic benchmarks

Multimodal benchmarks

Limitations

Kimi K2.6

Long-Horizon Coding

Coding-Driven Design

Agent Swarms, Elevated

Proactive Agents

Bring Your Own Agents

Benchmark Table

Footnotes

Agent Swarm

The Limits of Single-Agent Reasoning

Scale Out, Not Just Up

Agent Swarm Best Practices

Discovery at Scale

Output at Scale

Perspective at Scale

Now Available to Top Tier Subscribers

WorldVQA

Introducing WorldVQA

Overview

The Dataset

Distribution of Tasks per Category

Using WorldVQA to compare models

Measuring Calibration: Confidence vs. Accuracy

Conclusion

Kimi K2.5

1. Coding with Vision

2. Agent Swarm

3. Office Productivity

4. Conclusion

Appendix

Benchmark table

Footnotes

Kimi Vendor Verifier

Rebuilding the "Chain of Trust": Kimi Vendor Verifier

Official Evaluation Results

Why We Built KVV

Our Solution

Testing Cost Estimation

An Open Invitation

Kimi K2 Thinking

Evaluations

Agentic Reasoning

Agentic Coding

Agentic Search and Browsing

General Capabilities

Inference Efficiency

Full Evaluations [2]

Kimi-K2-Instruct-0905

1. Model Introduction

Full Evaluations ^[2]