Kimi K3.1 Pro: Sparse Reads, Not a Smaller History

A concrete change from K3

As of 5 November 2026, Moonshot's Kimi-K3.1-Pro repository contains downloadable weights in 96 safetensors shards. This guide pins the checkpoint to d9e99b0792a5e93b78ea9bea62a68167e36ddd9e and SGLang source to 93e6b42d8afa99702547eeeb340986884c9616a3. It is a released model, not an inference inferred from an announcement.¹²

The relevant change from our Kimi K3 architecture guide is precise: the 24 Gated MLA layers now use DeepSeek Sparse Attention (DSA). The 69 Kimi Delta Attention (KDA) layers remain recurrent. Sparse global reads complement recurrent summaries; they do not turn the global cache into a fixed-size summary.³² No model inference, quality or throughput benchmark was run for this article.

What changes, and what does not?

Residual width remains 7,168, with 93 layers. One-based global layers are 4, 8, …, 92, 93; the other 69 use KDA. The text configuration retains KV latent rank 512, query latent rank 1,536, 96 main attention heads, and 128-wide value heads. It also retains block Attention Residuals with block size 12 and LatentMoE width 3,584, 16 of 896 routed experts and two shared experts. These inherited mechanisms are explained in the K3 guide; DSA does not replace the MoE router.²

The new indexer has 32 query heads of width 128, one shared 128-wide index key per stored position, and a 2,048-position selection budget. This is a separate, cheaper representation for choosing positions, not the main attention's compressed K/V. The pinned SGLang configuration explicitly requires index_topk_freq = 1: every global layer runs its own indexer. Do not import cross-layer index reuse from another DSA model.⁴⁵

Follow one query through the indexer

For $T$ query positions and $S$ eligible cached positions, the indexer projects the query latent into [T,32,128]. A different projection maps residual states to [S,128] keys, followed by LayerNorm; another maps each query's residual state into 32 head weights. Their scoring path produces position scores, then selects up to 2,048 causal candidates per query. The resulting position-ID array is logically [T,2048], with invalid/short-prefix entries handled by the backend.⁵

The adapter matters: K3.1 specifies index_rope_dim = 0, LayerNorm rather than RMSNorm for index keys, FP32 key normalization, no Hadamard transform, and a BF16 head-gate path. These are explicit implementation choices, not defaults to borrow from DeepSeek's indexer. The indexer and main attention are also distinct from expert routing: one chooses positions, the other FFNs.⁴

The main global attention remains a gated MLA implementation with skip_rope=True. NoPE here means no rotary transform in that path, not “all configured rotary-width coordinates disappear.” The constructor still passes a 64-wide companion key component alongside the 512-wide latent. The indexer's zero rotary width is a separate setting.⁶

After sparse attention, SGLang applies a learned coordinate-wise sigmoid output gate before the output projection. Neither the indexer score nor that gate is a probability that a retrieved fact is correct. The KDA, global attention, depth mixing and expert routing retain different roles and states.

The math of sparse attention and its missing mass

Let $s_{tj}$ denote the learned indexer score for query $t$ and eligible position $j$. Let $I_t$ contain its highest-scoring up-to-2,048 positions. For one main attention head, define its different attention logit as $a_{tj}$. The sparse read is a softmax restricted to $I_t$:

This notation abstracts the absorbed MLA projections and floating-point kernels; it is not a claim that the runtime materializes all head-specific K/V. Selection scores $s$ are not main attention logits $a$. High indexer rank therefore does not mathematically guarantee high main-attention mass.

An illustrative bound makes that limitation explicit. Hold logits and values fixed and compare with a hypothetical full read. If $\epsilon$ is the full softmax mass outside $I_t$, then:

The proof uses only that both retained and dropped reads are convex combinations of bounded values. It says small omitted mass can bound one head's local read error. It says nothing about whether this indexer actually keeps that mass. It also does not bound the final token distribution after output projection, residual mixing and later layers, or compare two independently trained checkpoints. No empirical value of $\epsilon$ was measured here.

Why top-2048 does not mean a 2048-token cache

At the configured 1,048,576-token context ceiling, 2,048 positions are 1/512, or 0.1953125%, of that length. This is an arithmetic comparison of the final selection width, not a 512× end-to-end speedup. Index scoring and top-k still operate over eligible history, and their keys must remain available for a different future query. Prefix prefill, recurrent work, MoE traffic and kernel scheduling remain additional.²⁵

A transparent reference budget illustrates the difference. Suppose one request retains, per global layer and position, a 512+64-element MLA representation and a 128-element index key, all in BF16. Then:

That hypothetical payload is 33 KiB/token, or 33 GiB at the full configured length. It is not a measured or universal runtime allocation: index/MLA quantization, scale metadata, padding, sharding, replication, offload and page allocation change physical memory. The essential point is the dependence on $S$, not the particular dtype. KDA recurrent/convolution state, prefix checkpoints, weights and workspaces are additional. See the KV-cache budget guide.

Do not multiply the 512-wide shared latent by all 96 main heads as if these were ordinary independent head-specific KV vectors. Conversely, do not assume MXFP4 expert weights imply a four-bit cache. The checkpoint's quantization configuration has kv_cache_scheme = null; weight and cache precision are separate decisions.²⁶

What a deployment must validate

The checkpoint excludes attention, shared experts, vision/projector and several other projections from its MXFP4 weight group. Active parameter count is not resident checkpoint size. Pin the engine and verify that its loader instantiates KimiK31ForConditionalGeneration, reads the nested DSA configuration and loads the indexer parameters. The pinned SGLang model has an explicit check for missing indexer weights; silently running a dense K3 path would not reproduce this architecture.²⁶

The official vLLM recipe distinguishes baseline serving from optional DCP, cache precision and speculative-decoding configurations. Its synthetic-acceptance throughput mode skips real verification and is explicitly unsuitable for serving users or evaluating accuracy. That is a benchmarking shortcut, not a valid decoder setting. The recipe also says long-context accuracy was not measured for its NVFP4 MLA-cache option. Neither option should be presented as a quality-preserving default.⁷

No throughput figures are reproduced here. To evaluate a deployment, hold hardware, concurrency, precision, input/output lengths and quality criteria fixed. Measure indexer time separately from sparse attention, peak memory, prefill latency, decode latency and long-context retrieval/agent success. Store all state at the same prefix boundary when reusing KDA and global caches. A one-million-token limit is not a recall guarantee.

Practical consequence

K3.1 Pro is worth considering when long-lived agent sessions make reading a large global history expensive while recurrent layers supply compact context mixing. That is a mechanism-based hypothesis to test, not a recommendation based on a launch score. It does not make all expert weights or old cache records disappear.

Compared with Step-5's sparse GQA, the retained global representation here is MLA-compressed and paired with KDA. Compared with DeepSeek V4.1 Flash, do not assume the same compression or index-reuse policy. The useful lesson is to budget selection work, selected reads, retained history and approximation quality separately.

Primary sources