GLM-5.3 MTP on B300: Acceptance Math and NCCL NVLS

Multi-token prediction (MTP) is an acceptance-and-cost problem. This article uses a GLM-5.3 deployment on eight B300 GPUs (Blackwell Ultra, sm_103)¹ as a configuration example. The rates in the worked table are illustrative inputs, not measurements. It also describes documented NVLink-SHARP (NVLS) behavior across NCCL versions.

The setup

The GLM MTP head ships inside the checkpoint - one extra prediction layer, not a separate draft-model download. GLM-5.3's published configuration declares a sparse-attention MoE (GlmMoeDsaForCausalLM), a 1,048,576-token context limit, and one MTP prediction layer². vLLM (speculative_config={"method": "mtp", "num_speculative_tokens": k})³ and SGLang (--speculative-algorithm NEXTN)⁴ can use it. The vLLM GLM-5.3 recipe shows an eight-GPU FP8 setup with TP=8, MTP k=5, and --kv-cache-dtype fp8_e4m3; a separate NVFP4 variant quantizes routed experts while retaining BF16 in attention, shared experts, and dense layers⁵. These are documented configurations, not results from a benchmark in this repository.

Acceptance math: tokens per step, not throughput

Acceptance probabilities determine the expected token count per verification step; draft and verifier costs also determine throughput. Expected tokens per step is not k plus one. If $a_j$ is the probability of accepting draft position $j$ conditional on all earlier positions being accepted, then the probability of accepting through position $i$ is $p_i=\prod_{j=1}^{i}a_j$. The framework is speculative decoding as introduced by Leviathan, Kalman and Matias: draft tokens are verified against the target model, and accepted tokens are resampled under the target distribution⁶:

vLLM's log_stats lines can report Mean acceptance length, accepted-vs- drafted token counts, and Per-position acceptance rate⁷. The following joint probabilities $p_i$ are hypothetical. They illustrate the arithmetic and must not be used as a GLM-5.3 performance claim:

On this example chain, k=2 yields ~64% of the k=5 tokens per step, k=3 ~77%. These ratios are not speedup measurements. They depend on the per-position rates your workload actually produces. Public measurements show a spread across datasets and positions for GLM MTP models, and the vLLM recipe for the smaller GLM-4.5 recommends num_speculative_tokens 1 for optimal throughput on that model⁸; the GLM-5.3 recipe shows k=5 as a configuration example⁵.

Deeper speculation buys marginal tokens while every draft position adds cost:

• Wasted verify work: a rejected draft still consumed a verify slot.
• Discarded later drafts: when a position is rejected, later positions in that draft cannot be accepted. Their generation and verification can consume work without adding output tokens.
• KV-cache pressure: num_speculative_tokens reserves KV entries for draft positions per request; vLLM docs recommend starting small³.

Depth is a per-workload decision driven by measured acceptance - public benchmarks find acceptance varies wildly across datasets, requests, and positions⁸. Measure first, then set k - the log_stats lines named above are the measurement.

NVLS failure behavior depends on NCCL version

NVLS initialization failures have not always produced the same outcome: some NCCL releases fall back to another collective, while others return an error. Check the behavior of the installed release.

The mechanism, from the documentation:

• NVLS is NVLink SHARP: in-network reduction offloaded into the NVSwitch domain; NCCL supports it since 2.17 on third-gen NVSwitch with Hopper or later⁹.
• Fallback in NCCL 2.27.3: NCCL_NVLS_ENABLE defaults to 2 (auto-detect). That release added graceful fallback when NVLS initialization fails; setting NCCL_NVLS_ENABLE=1 preserved an error instead¹⁰.
• A documented hard failure in NCCL 2.29.7: issue #2077 reports multicast-slot exhaustion on an eight-H200 system. The reporter observed a fatal ncclUnhandledCudaError with NVLS enabled and a successful run with NCCL_NVLS_ENABLE=0. The report describes a 128-slot hardware limit; it does not establish the behavior of every NVLS failure on every NCCL release¹¹.
• The stack below NCCL is version-locked: Fabric Manager checks the loaded kernel driver stack at initialization and aborts on mismatch; HGX B200/B300 nodes (fourth-gen NVSwitch) additionally require the NVLSM service, from the driver-branch-matched nvidia-open and nvlink5 packages¹². The fabric stack moves as a set by NVIDIA's own packaging.

The working rules that follow from the documentation:

1. Pin driver + fabric-manager + NCCL as a set - FM is version-locked to the driver by design¹².
2. Probe with a tiny all-reduce benchmark before blaming the serving stack.
3. Use NCCL's INFO logs, including the relevant collective and NVLS debug subsystems when available, to inspect algorithm selection¹³; confirm the result with the measured probe rather than assuming the fast path.

Transferable rule: verify the fast path and the failure behavior on the installed NCCL version.

Notes on output fidelity

Exact speculative decoding with the appropriate verifier and rejection sampling preserves the target model's output distribution⁶. It does not promise the same token sequence as a separate stochastic target- only run. Any serving implementation must be checked for the exactness assumptions used by that result.