Moonshot AI Launches Kimi K3 Model with 2.8 Trillion Parameters and Quantized Open Weights

Chinese artificial intelligence startup Moonshot AI released the open-source weights for its flagship Kimi K3 model on July 27, 2026, following its initial API launch on July 16. At 2.8 trillion total parameters, the autoregressive Mixture-of-Experts transformer is the first open-weights model to enter the three-trillion parameter class. The model features a one-million-token context window, native multimodal vision, and optimized architectural designs to reduce the high compute demands of large-scale local deployment.

Inside the 2.8-Trillion-Parameter Mixture-of-Experts Architecture

Artificial intelligence models utilize parameters, which are mathematical variables, to process data and generate outputs. Typically, larger parameter counts allow models to capture more complex patterns but require massive computational resources. Kimi K3 addresses this compute hurdle through a Mixture-of-Experts architecture, which divides the 2.8 trillion parameters into 896 specialized sub-networks, or experts.

During inference, which is the process of generating a response, the model does not activate all 2.8 trillion parameters. Instead, Moonshot AI designed the model to activate only 16 of the 896 experts for each token processed. This routing mechanism means that only 104.2 billion parameters are active at any given moment, significantly reducing memory and processing requirements.

By making the weights of this model open-source, Moonshot AI allows developers to download the model’s actual learned mathematical parameters. This differs from closed-source systems where users must query a model over a paid web interface controlled by the developer. Open-weights models grant teams the flexibility to run the model on their own server infrastructure, customize the model for specific tasks, and guarantee data privacy by keeping operations offline.

Mitigating Compute Overhead with Stable LatentMoE and Quantization

Running a three-trillion-class model locally typically requires extensive high-end hardware, which can be cost-prohibitive for smaller operations. To make hosting Kimi K3 viable, the model uses a framework called Stable LatentMoE. This system uses latent-space routing to project token activations into a narrower mathematical dimension for expert computation, which lowers the physical bandwidth needed to transfer weights between graphics processing units.

The framework also incorporates a technique called Quantile Balancing. This optimization replaces traditional heuristic routing updates with expert allocation derived directly from router-score quantiles. According to the company’s technical reports, this keeps expert utilization balanced without introducing unstable training loops.

To further decrease the hardware footprint, Moonshot AI utilized quantization-aware training. This process reduces the numerical precision of the model’s data from standard high-precision formats to lower-precision representations. Kimi K3 is trained with MXFP4 weights and MXFP8 activations, which significantly decreases the amount of Video RAM required to store and run the model on server racks.

To assist with self-hosting, Moonshot AI released integration support for inference runtimes such as vLLM and SGLang alongside the model weights. These systems act as open-source execution engines that optimize how hardware resources process large language model queries.

Bypassing Quadratic Cost with Kimi Delta Attention and Attention Residuals

Large language models traditionally use attention mechanisms to determine how words or pixels relate to one another in a sequence. Under standard quadratic attention, the compute cost scales quadratically with the length of the input, making long contexts extremely expensive to run. Kimi K3 uses a hybrid system called Kimi Delta Attention (KDA) to manage its one-million-token context window.

KDA is a linear attention mechanism that replaces standard quadratic attention in three out of every four layers in the model’s architecture. Rather than scaling quadratically, KDA functions as a recurrent process that decays memory on a channel-by-channel basis instead of across entire attention heads. This fine-grained gating lets specific semantic features persist over long sequences while irrelevant details decay quickly.

The model also replaces standard residual connections with a mechanism called Attention Residuals. Under traditional layouts, layers process and accumulate information step-by-step directly from the preceding layer. Attention Residuals allow deeper layers in Kimi K3 to selectively retrieve representations from arbitrary earlier layers, a design that Moonshot AI states provides approximately 2.5 times the overall scaling efficiency of its prior model, Kimi K2.

The Kimi K3 Custom License and Commercial Revenue Thresholds

While Moonshot AI has made the model’s weights free to download, the model is governed by a custom usage license rather than standard open-source licenses like Apache 2.0. The Kimi K3 license functions similarly to an MIT license for individual researchers, allowing them to copy, modify, and distribute the model. However, it imposes strict revenue and usage thresholds on large-scale commercial entities.

If a company or any of its affiliates operates a Model as a Service business—such as providing third-party API access to language model inference—and generates more than $20 million in aggregate revenue over 12 consecutive months, it cannot use the model for commercial purposes under the free tier. Such organizations are required to enter into a separate, paid commercial agreement directly with Moonshot AI.

The license also mandates that any service provider exceeding 100 million active users or generating over $20 million per month in revenue must visibly display the Kimi K3 brand. This licensing structure prevents competing commercial cloud platforms from offering Kimi K3 as a managed service without entering a paid commercial agreement.