DSpine for Qwen3.8-27B: Causal Features Through Depth

A released drafter, not a replacement language model

Tencent publishes Qwen3.8-27B-DSpine-b16 weights and a usable SGLang plugin. It accelerates an existing target through speculative decoding: propose a continuation cheaply, let the target verify it, and commit only the accepted prefix plus a target correction or bonus token. This is not a standalone 27B model and not a generic drafter for arbitrary targets.¹²

The new mechanism is adjacent causal feature injection. Positions are processed together, but each draft layer writes a prediction of the preceding position's features into its successor. The release also has a serial last-write refinement after the parallel Transformer. Calling the entire proposal pipeline “fully parallel” would hide that second stage. The Nemotron DFlash guide explains the underlying block-speculation idea; this article focuses on what DSpine adds.

Checked 12 November 2026: draft revision 5f54174d301dbca65d9aa4448dc47e56732d9a12, target revision 37561816e66b5b9153b1b64c10968c18c1e6f4a0, plugin revision c66bf4405260451a32161205666048c60ca1a95f and SGLang 0.5.22, commit baa9befa381511cd4d56f83980951e5a45fffc6d. The plugin explicitly pins that SGLang release because it subclasses internal APIs. Tensor counts below are source-derived payloads, not GPU measurements. No model inference or throughput benchmark was run.³⁴⁵

What actually enters the draft block

The block has 16 rows: one known anchor and 15 proposal positions. Five selected target feature IDs, 1, 16, 31, 46 and 61, supply context. Each target vector has width 5,120; the inherited DFlash path concatenates the five vectors into width 25,600, projects to 5,120 and applies a learned RMSNorm before building draft context K/V.³⁶

The drafter has five dense SwiGLU layers, hidden width 5,120, FFN width 17,408, 32 query heads, eight KV heads and head width 128. Thus its query projection is 4,096-wide, not 5,120-wide. All five layers use an 8,192-position causal attention window with RoPE base 10,000,000. This differs from the target's 64-layer GDN/full-attention hybrid. Do not transfer the target's attention geometry to the drafter.³⁴

Unlike the non-causal draft block in the earlier E4B example, this checkpoint explicitly enables causal draft-block attention. It also does not become ordinary token-by-token decoding: future proposal positions still start from placeholders and are processed together, rather than waiting for discrete tokens sampled by earlier positions.⁷⁶

The bridge: read 1,024 features, write through rank 512

After each draft layer, let $x_{\ell,t}\in\mathbb R^{5120}$ be the combined post-MLP residual state. The bridge first computes its RMS magnitude $r_{\ell,t}$ and predicts a code vector of width 1,024. Ignoring BF16 rounding and writing the small numerical floor implicitly, its read is:

Here $R_\ell$ is [1024,5120], with a separate read matrix per layer. For the anchor row, the implementation replaces the predicted code with $\sqrt{1024}$ times the known anchor token's stored codebook vector. That table is [248320,1024]. It is a learned token-feature representation, not an external document index or an n-gram Engram lookup.⁷⁸

Each receiving row combines its own code with the immediately preceding row's code. Three shared projections map 1,024 coordinates to 512: a receiver gate $G$, predecessor gate $G_p$ and predecessor-value map $A$. Each layer has its own write matrix $W_\ell\in\mathbb R^{5120\times512}$:

The message is a gated prediction, not a confirmed earlier token. Its update scales with the receiver's RMS magnitude; a standard learned RMSNorm then prepares the next sublayer input. The kernel zeros the write at each block's anchor, preventing a predecessor read from another request from becoming a cross-request update.⁸

Every row reads predecessor codes from the same completed layer, so those writes can run in parallel. The next layer sees the updated states. The bridge therefore propagates local causal hints through network depth without rerunning a whole Transformer for every draft token. This is an explanation of information flow, not a proof of longer acceptance. Attention supplies additional causal paths; the bridge's adjacent edges alone do not define the network's entire receptive field.

The last write is refined using discrete candidates

The release sets head_topk = 16 and refines only the last write. It first retrieves a candidate set for each of the 15 proposal rows. The optimized CUDA-graph path scores the full vocabulary approximately with a row-scaled FP8 head, takes 128 candidates, rescores those using the original BF16 head, and retains 16. A fallback path directly multiplies by the BF16 vocabulary head.⁹¹⁰

Rescoring does not prove that FP8 retrieval contained the global BF16 top 16. It is candidate generation for a proposal, not an authority on target output. For each receiver candidate, the tail evaluates the last bridge write conditioned on a discrete predecessor candidate instead of only its predicted continuous feature. With 16 predecessor and 16 receiver candidates, this gives a small 16 × 16 conditional score table per position, followed by a serial chain through the positions. It does not rerun the five Transformer layers for every branch.⁸¹⁰

The first proposal is conditioned on the real anchor. Later selections depend on the preceding selected candidate. Greedy decoding walks the best-candidate table; sampled decoding constructs conditional probabilities, draws the chain and retains those probabilities for verification. The implementation has distinct greedy and sampled paths; the sampled path retains the normalization of each refined state. Treating greedy lookup scores as a universal probability table would be incorrect. The sampled tail uses FP64 factor calculations with a direct fallback, while the sampler enforces true FP32 matrix-multiply precision with TF32 disabled.¹⁰¹¹

This creates two different causal mechanisms: continuous feature messages during depth, then discrete candidate conditioning at the tail. Both are cheaper surrogates for repeated target execution, but both can make poor proposals.

Why a restricted proposal need not restrict target quality

For speculative sampling, $q$ must be the distribution that actually produced the proposed token, including candidate restriction, temperature and filtering. Verification compares it with the target's deployed distribution $p$:

Accepted mass is $\min(p,q)$; the rejection branch contributes the missing positive residual. Together they reconstruct $p$ under the exact algorithm. A target token absent from the draft's 16 candidates can still be emitted through correction. Candidate restriction can reduce acceptance without mathematically restricting the final target distribution.¹²

That guarantee has implementation conditions. The plugin retains the current round's sampled $q$, implements the target backend's filtering semantics, and has numerical tests comparing sparse and dense verification with the same probabilities and random coins. Those are published tests, not a claim that we executed CUDA validation. Same target distribution also does not promise identical text for the same seed, or identical numerics across hardware. The target's GDN state must be committed only through the accepted prefix; the release provides ReplaySSM-specific commit handling.¹⁰¹³

An attention window is not a cache-size guarantee

For five draft layers, eight KV heads, 128 coordinates and BF16 storage, each retained position has 20 KiB of logical draft K/V:

An ideal compact 8,192-position window would contain 160 MiB per sequence. But this plugin explicitly requires SGLang's dense BF16 MHATokenToKVPool and rejects bounded/hybrid draft pools as unvalidated. The 8,192-position attention mask bounds reads; it is not proof that older K/V storage is reclaimed. For scale, retaining 131,072 positions would mean 2.5 GiB of draft KV payload, before target state, paging, buffers or prefix sharing. This is conditional arithmetic, not an observed allocation.¹⁴¹⁰ Compare the cache-layout discussion.

The bridge itself has costs beyond five Transformer layers. Its BF16 token-code table contains 485 MiB of payload. The greedy tail precomputes tables shaped [248320,512] and [248320,1024], together another 727.5 MiB in BF16. They accelerate repeated candidate scoring, but are additional resident tensors. FP8 retrieval adds a head copy and scales; sampled refinement has separate factors and workspaces. None of these counts is the complete serving footprint.⁷⁸⁹¹¹

Where the released serving path stops

For this Qwen checkpoint, the audited plugin requires CUDA, TP=1, block size 16, an unquantized contiguous BF16 target output head, BF16 draft activations/weights, FA3 draft attention and at most eight concurrent requests. It rejects prefill/decode disaggregation and the named FP4 KV formats. The ReplaySSM recipe additionally requires PP=1, FP32 recurrent state, Triton linear attention and the extra-buffer prefix-cache strategy. A target-weight quantization recipe is therefore not permission to quantize the shared output head or the draft arbitrarily.¹⁵¹⁰

The README's 131,072-position serving command and the config's 262,144 maximum-position field are different scopes, not interchangeable tested capacities. The separately published MLX integration is not evidence that this CUDA kernel path or its numerical properties transfer unchanged to Apple hardware.³²

What to measure before using it for agents

The model card compares DSpine with block size 16 against DFlash2 with block size 8, using SGLang 0.5.22, BF16, FA3 and concurrency eight; DSpine also uses dynamic verification and ReplaySSM. Agent results replay tool-call prefixes from 20 conversations per subset. That is a serving-system comparison, not an isolated ablation of the predecessor bridge. Its ratios are not repeated here as expected performance on another GPU or workload.¹

For your own long answers or tool-using agents, measure end-to-end decode time, accepted-prefix length, draft/tail/verification time, peak memory and cancellation behavior with the same target, context, sampling and concurrency. Acceptance length is not speedup: if one cycle costs $C_d+C_v$ and commits an average $\tau$ tokens, its cost per output token is $(C_d+C_v)/\tau$, before any separately accounted scheduling overhead. Here $C_d$ includes both the draft forward and its refinement tail.

The useful release detail is not “parallel prediction removes causality.” It is a two-stage approximation to causal conditioning, protected by target verification and accompanied by explicit memory and backend constraints. Whether it pays for itself remains a workload-specific measurement.

Pinned primary sources