Kimi-K3 Arrives: Moonshot's Open 2.8-Trillion-Parameter Model

Moonshot AI has released Kimi-K3, a 2.8-trillion-parameter open-weight multimodal model the company calls its "most capable model to date." At the same time, the lab published Kimi-K2.7-Code, a coding-specialized model that, per its model card, is "built upon Kimi K2.6" with meaningful improvements in agentic software-engineering performance. These two releases together mark the most ambitious capability push yet from the Chinese AI lab, putting moonshotai kimi squarely in contention with frontier closed-source systems. Developers already running moonshotai kimi k2 instruct, moonshotai kimi k2 thinking, or earlier checkpoints like moonshotai kimi k2 instruct 0905 will find both releases directly relevant to their upgrade path.

What Kimi-K3 Is — And Why 2.8 Trillion Parameters Matters

According to the official Hugging Face model card, Kimi-K3 carries 2.8 trillion total parameters but activates only 104 billion per token through a Mixture-of-Experts (MoE) architecture. That sparse-activation design is what makes this scale feasible: 896 experts are defined, and each token routes through just 16 of them plus 2 shared experts, keeping per-token compute tightly bounded while overall model capacity stays enormous. Moonshot AI explicitly claims this makes Kimi-K3 "the world's first open 3T-class model." A caveat on the "open-weight" framing: the weights are downloadable, but K3 ships under a custom Kimi K3 License rather than an OSI-approved open-source license, so "open-weight" here means publicly available rather than freely redistributable.

The sheer parameter count isn't just a vanity metric. In MoE systems, total capacity governs how much knowledge and behavioral diversity can be encoded across the expert pool, while activated parameters determine inference cost. At 104B activated, Kimi-K3 sits in roughly the same inference-cost tier as large dense models — but draws on a knowledge pool about 27× larger per forward-pass selection. For teams deploying on owned or rented hardware, the practical constraint is memory bandwidth across the full 2.8T weight set, not floating-point compute alone. A model this size requires a large GPU cluster or high-memory server configuration; it won't run on a single desktop node.

A Genuinely New Architecture: KDA, AttnRes, and Stable LatentMoE

Kimi-K3 is not a scaled-up version of its predecessors. Moonshot AI introduced three novel architectural components that together set it apart from both moonshotai kimi k2 and the broader open-source landscape.

Kimi Delta Attention (KDA)

Of the model's 93 transformer layers (1 dense plus 92 MoE), 69 use a new mechanism called Kimi Delta Attention (KDA), while the remaining 24 use a Gated variant of Multi-head Latent Attention (MLA). KDA is listed alongside a complementary construct called Attention Residuals (AttnRes). The model card doesn't disclose the full mathematical formulation for either. Based on the naming convention, KDA likely operates over a delta (difference) representation of hidden states rather than raw activations — a technique that can reduce the effective rank of the attention matrix and improve gradient flow at scale — but this interpretation is our own inference and remains speculative until a formal technical report appears.

Stable LatentMoE

The MoE routing layer is built on what Moonshot calls a Stable LatentMoE framework. The company claims this achieves roughly a 2.5× improvement in overall scaling efficiency over Kimi K2 — a figure the card provides no underlying data or methodology for, so it is worth treating as a vendor claim until independently reproduced. Each expert has a MoE hidden dimension of 3,072, with the model's overall hidden dimension at 7,168 and 96 attention heads. The activation function is listed as SiTU-GLU, a variant replacing the SwiGLU used in prior Kimi generations; the card gives no further definition of SiTU-GLU. The "stable" qualifier matters: MoE models historically suffer from expert collapse and load imbalance at scale, problems that erode effective capacity and make training brittle. A training-stable routing mechanism would be a genuine engineering achievement, though the full mechanism remains undisclosed.

Native Multimodality with MoonViT-V2

Vision is handled natively by MoonViT-V2, a 401-million-parameter encoder that feeds directly into the main transformer. This is a small size upgrade from the 400M MoonViT encoder used in Kimi-K2.7-Code and K2.6, but the integration is architecturally deeper: text and image inputs are processed within a single unified model (the card lists the modality as text and image) rather than through an adapter or late-fusion approach. That native multimodality is a meaningful shift from earlier text-first Kimi generations, which treated vision as a bolt-on capability.

Full Technical Specifications at a Glance

Attribute	Kimi-K3	Kimi-K2.7-Code
Total Parameters	2.8T	~1T (1.1T per safetensors)
Activated Parameters / Token	104B	32B
Layers	93 (1 dense + 92 MoE)	61 (1 dense)
Attention Mechanism	69× KDA + 24× Gated MLA	MLA
Number of Experts	896	384
Experts per Token	16 + 2 shared	8 + 1 shared
Attention Heads	96	64
Context Length	1,048,576 tokens (1M)	256,000 tokens (256K)
Vision Encoder	MoonViT-V2 (401M params)	MoonViT (400M params)
Activation Function	SiTU-GLU	SwiGLU
Quantization	MXFP4 weights / MXFP8 activations (QAT)	INT4 (27 variants)
License	Kimi K3 License (custom/proprietary)	Modified MIT
HF Downloads (approx., at time of writing)	~2,850	~695,700

Benchmark Performance: Where Kimi-K3 Lands

Moonshot AI published benchmark comparisons in the official model card. All Kimi-K3 scores were measured at reasoning_effort = "max" and temperature = 1.0 (with top-p = 0.95 for single-step tasks and top-p = 1.0 for agentic tasks). Important caveat: the card compares K3 against models labeled Claude Fable 5, GPT-5.6 Sol, Claude Opus 4.8, GPT-5.5, and GLM-5.2 — version strings that do not correspond to publicly confirmed releases we can verify as of this writing. Treat the specific comparison scores as self-reported vendor figures and wait for independent third-party evaluation before drawing firm conclusions about cross-model rankings. Benchmark names below (including ProgramBench, FrontierSWE, SWE-Marathon, and MLS-Bench Lite) are reproduced as labeled on Moonshot's card; some are not yet widely established, so cross-lab comparability is limited.

Reasoning and Knowledge

On GPQA Diamond, Kimi-K3 scores 93.5, placing it among the highest-reported open-model results on this graduate-level science reasoning benchmark. On AA-LCR — a long-context reasoning benchmark that Moonshot's card attributes to the third-party evaluator Artificial Analysis (scores cited "as of July 23, 2026"), not to an internal Moonshot benchmark — Kimi-K3 reaches 74.7, versus 74.3 for GPT-5.5 and 71.3 for GLM-5.2 in the same table. On HLE-Full (Humanity's Last Exam, a demanding test of broad expert-level knowledge), the model achieves 43.5 without tools and 56.0 with general tool augmentation. That gap is substantial, and it highlights how much this model leans into agentic, tool-augmented settings.

Coding and Agentic Performance

The coding numbers are particularly relevant for developers. On ProgramBench, Kimi-K3 scores 77.8. On Terminal-Bench 2.1, it reaches 88.3. The model achieves 67.5 on DeepSWE, 81.2 on FrontierSWE, 42.0 on SWE-Marathon, and 72.9 on the in-house Kimi Code Bench 2.0. For agentic web and tool tasks, BrowseComp lands at 91.2, OSWorld-Verified at 84.8, and MCPMark-Verified reaches 94.5. MCPMark-Verified is a human-verified agentic benchmark that (per public MCPMark documentation) spans environments such as Notion, GitHub, Filesystem, Postgres, and Playwright. Moonshot notes that MCPMark-Verified tooling will be open-sourced soon, which will let the community replicate these figures.

Multimodal Benchmarks

On MathVision, Kimi-K3 scores 94.3 without Python tools and 97.8 with them. Video-MME (with subtitles) reaches 90.0, MMVU reaches 82.1, and OmniDocBench scores 91.1. These results reflect MoonViT-V2's native integration — vision is architecturally central here, not appended after language pretraining.

Kimi-K2.7-Code: The Precision-Tuned Coding Specialist

Running alongside Kimi-K3's general frontier ambitions, Kimi-K2.7-Code targets a narrower but commercially critical problem: the best open-weight model for complex, end-to-end software engineering workflows. Its model card states it is a "coding-focused agentic model built upon Kimi K2.6" — which itself iterated on moonshotai kimi k2.5 and the original moonshotai kimi k2. The model keeps the same ~1T-parameter, 32B-activated MoE architecture that made the K2 family popular with developers, but retrains the weights specifically for long-horizon coding tasks.

The headline improvement over K2.6 is a ~30% reduction in thinking-token usage, meaning agentic pipelines get more efficient reasoning without sacrificing output quality. On the in-house Kimi Code Bench v2, K2.7-Code jumps from K2.6's 50.9 to 62.0 — a gain of 11.1 points. On MLS-Bench Lite, a benchmark targeting the ability to generate novel machine-learning research implementations, the score rises from 26.7 to 35.1. For MCP tool orchestration, MCP Mark Verified climbs from 72.8 to 81.1, and MCP Atlas from 69.4 to 76.0. Program Bench improves from 48.3 to 53.6. Note that in Moonshot's own table, the labeled closed frontier comparators (e.g., "GPT-5.5," "Claude Opus 4.8") still lead on several of these — for example 69.0 and 67.4 respectively on Kimi Code Bench v2 — so K2.7-Code's gains are best read as intra-family progress rather than a claim of category leadership.

Crucially, the deployment surface is identical to moonshotai kimi k2.5 and K2.6: any infrastructure configuration already working for those models is directly reusable. Teams running moonshotai kimi k2 instruct or moonshotai kimi k2 thinking pipelines can upgrade to K2.7-Code with no architectural changes — just a weight swap. The original moonshotai kimi k2 instruct 0905 checkpoint established the instruction-following baseline that subsequent versions have refined, and K2.7-Code continues that lineage without breaking compatibility.

The model ships under a modified MIT license with 27 quantization variants available in the Hugging Face model tree, making it far more permissive than Kimi-K3's custom Kimi K3 License. Developers with commercial redistribution or fine-tuning requirements should pay close attention to that distinction. The much larger download volume for K2.7-Code (roughly 695,700 at time of writing) versus K3 (around 2,850) reflects a straightforward hardware reality: the K2 family runs on multi-GPU workstations, while K3 demands a cluster. These snapshot counts move constantly, so treat them as approximate.

Why the token-efficiency gain matters in practice: Kimi-K2.7-Code's ~30% reduction in thinking tokens isn't just a cost saving — it directly affects latency in agentic loops where thinking tokens accumulate across dozens of sequential tool calls. A 30% cut can mean the difference between an agent completing a multi-step task within a rate-limited API window and hitting a timeout, particularly in CI/CD or IDE-integrated workflows where response budgets are tight.

Quantization Strategy: Deploying at Scale Without Precision Collapse

One of the more technically significant details in the Kimi-K3 release is that quantization is baked into training rather than bolted on afterward. Rather than applying post-training quantization (PTQ) to a full-precision checkpoint, Moonshot used quantization-aware training (QAT) with MXFP4 weights and MXFP8 activations.

MXFP (Microscaling Floating Point) is a format supported by recent NVIDIA GPU architectures that packs more values per tensor element while preserving greater dynamic range than INT4, achieved through per-group exponent sharing. The practical implication is significant: the model card's stated parameter count and capability figures apply to the quantized model directly. There is no higher-precision baseline that the distributed weights have been degraded from — the Hugging Face artifact is the production model. This sidesteps the precision-loss concerns that often accompany PTQ at scale, where capability can degrade unpredictably on tail tasks.

Kimi-K2.7-Code takes a more conventional approach, offering native INT4 quantization (the same method used for Kimi-K2-Thinking) across 27 separate variants in its model tree. Teams can select the precision tier that fits their hardware budget — from higher-precision inference on high-memory nodes down to aggressively compressed variants for constrained deployments.

Loading Either Model with Transformers

Both models use the transformers library with trust_remote_code=True, enabling standard AutoModel and pipeline patterns. A minimal loading example for K2.7-Code:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "moonshotai/Kimi-K2.7-Code"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    device_map="auto",
    torch_dtype="auto",
)

For Kimi-K3, substitute "moonshotai/Kimi-K3" as the model_id. Keep in mind that K3's memory footprint requires appropriate multi-GPU or distributed inference configuration; device_map="auto" will attempt to shard across available devices automatically.

Key Takeaways

• Kimi-K3 is Moonshot's largest model to date at 2.8 trillion total parameters, activating 104B per token via a novel Stable LatentMoE architecture with 896 experts and 16 selected per token — described by Moonshot as "the world's first open 3T-class model," though it ships under a custom (not OSI-approved) license.
• Three new architectural innovations — Kimi Delta Attention, Attention Residuals, and Stable LatentMoE — differentiate K3 from every prior Kimi generation; Moonshot claims a 2.5× scaling-efficiency gain over moonshotai kimi k2, though this figure is self-reported, unmethodologized on the card, and awaits independent verification.
• A 1-million-token context window and native multimodality (text and image via MoonViT-V2) make K3 one of the most capable open models for long-horizon agentic tasks that require broad context retention.
• Kimi-K2.7-Code is built upon K2.6 (and shares the K2.6/K2.5/K2 architecture) without changing architecture, achieving an 11.1-point jump on Kimi Code Bench v2 over K2.6 and cutting thinking-token usage by ~30%.
• License divergence matters: K2.7-Code ships under a modified MIT license; K3 carries a custom Kimi K3 License. Developers with commercial redistribution needs must review K3's terms before deployment.
• The deployment gap is real: K2.7-Code's ~695K downloads versus K3's ~2,850 (approximate snapshots) reflect hardware realities. Most engineering teams will reach for K2.7-Code first; K3 remains a cloud or HPC-cluster deployment for now.
• MXFP4/MXFP8 QAT on K3 means the quantized artifact is the primary model — no post-training precision degradation to worry about in the standard distribution.
• Benchmark comparisons pit K3 against labels like Claude Fable 5, GPT-5.6 Sol, Claude Opus 4.8, GPT-5.5, and GLM-5.2 — version strings we cannot independently confirm as public releases — and the long-context AA-LCR figures are drawn from third-party Artificial Analysis data, not an internal Moonshot benchmark; treat cross-model scores as vendor-reported until independent replication is available.

The Road Ahead: What Comes After K3 and K2.7-Code

The moonshotai kimi model family has moved through point releases at a striking pace — from the original moonshotai kimi k2 instruct 0905 checkpoint through moonshotai kimi k2.5, moonshotai kimi k2.6, Kimi-K2.7-Code, and now the architectural leap to K3, all within a compressed timeline. Several signals in Moonshot's model cards point toward near-term continuations:

• MCPMark-Verified tooling is described as "to be open-sourced soon," which will enable community replication of the agentic benchmark scores and spur third-party fine-tuning efforts.
• Preserved thinking history mode in K3 suggests active work on multi-turn agentic reliability — a capability that matters enormously for long-running autonomous agents that must maintain coherent reasoning state across many tool-call steps.
• The logical next releases are either a K3-Code variant — applying K3's architectural innovations to the coding-specialist fine-tuning regime — or a distilled mid-size model that brings K3's KDA and Stable LatentMoE designs down to a scale (e.g., 200–400B total parameters) tractable on consumer GPU clusters.

For most development teams, the immediate action item is evaluating Kimi-K2.7-Code against existing moonshotai kimi k2 instruct or moonshotai kimi k2 thinking pipelines. The upgrade path is unusually frictionless — identical architecture, identical tooling, weight swap only — and the benchmark gains on coding and MCP tool orchestration are substantial within the Kimi family. Teams operating at cluster scale with multimodal or extreme long-context requirements should start planning for Kimi-K3 evaluation, paying close attention to the Kimi K3 License terms and the MXFP4 hardware compatibility requirements of their inference stack.