Liquid d1-3B: Branching Decisions Without Decoding

Liquid AI's d1-3B is a released 3.12-billion-parameter text-and-image decision model. It accepts one state and typed questions, then returns yes/no, choice or score probabilities without generating an answer token by token. The interesting part is not a new classifier head: the release reuses an LFM vocabulary head and changes how several prompts sharing the same state are evaluated.¹²

The custom implementation represents the shared state as a trunk and every question as a separate branch. Token-wise layers process the packed tree once. Convolution branches inherit the trunk's final inputs; attention branches see the trunk and their own earlier tokens, but never another question. This preserves the mathematical result of independent prompts while avoiding repeated state work. This article analyzes revision 051bcc464b01b9f92942b364d9586b0ef5912432, checked on 12 October 2026.³⁴

The released model, not the experimental sibling

d1-3B is built from the decoder-only LFM2.5-VL-3B backbone. Its published configuration and tensor metadata expose the following concrete shape.⁵³

The separate d1-omni-600M release is explicitly described by Liquid AI as experimental and under active development. It is therefore not used here as a production reference. d1-3B has downloadable BF16 and official GGUF artifacts plus a runnable PyTorch implementation. That makes the mechanism inspectable, although it does not guarantee that every runtime has an optimized tree kernel.¹⁶

For comparison, Cloudflare Clef performs one backbone prefill followed by a learned joint schema head. d1-3B instead keeps the ordinary tied vocabulary projection and evaluates one prompt branch per question. Both avoid an autoregressive answer loop, but the dataflows are different.

The LFM block underneath the decision API

Let $X\in\mathbb{R}^{B\times T\times2048}$ be the incoming residual state. Every decoder layer applies RMSNorm, either a short convolution or full attention, a residual addition, then RMSNorm, an MLP and another residual addition.

In a full-attention layer the projections reshape to the following tensors:

Grouped-query attention shares one KV head among four query heads. The persistent K/V payload of an ordinary decode cache would consequently be $2\cdot8\cdot64\cdot2=2048$ bytes per cached position and attention layer in BF16, before allocator overhead. d1's decision path does not run an autoregressive decode phase, so it does not retain a growing decode cache. It still performs the prefill attention and materializes temporary Q/K/V and score-related buffers.⁷

A convolutional layer projects each token to three 2,048-wide streams $b,c,u$. It applies a depthwise causal kernel of width three to $b\odot u$, gates the result with $c$, and projects back to 2,048. It has a two-token receptive state at a branch boundary. Across layers, the intervening MLPs and later attention layers still provide richer context; “kernel width three” is not the model's overall context length.

One trunk, many independent branches

For a state prefix of length $P$ and question suffix lengths $q_1,\ldots,q_N$, separate evaluation processes

token positions. The tree packs [trunk | branch 0 | … | branch N−1], so token-wise projections, norms and MLPs process

Branch positions restart at $P$. A width-three convolution prepends the trunk's last two inputs to every branch before applying the same causal filter. Therefore branch $n$ receives exactly the local history it would have in the independent sequence [trunk | branch n], rather than the tail of branch $n-1$.⁴

Attention is split into three calculations: causal trunk-to-trunk attention, branch queries over all trunk keys, and causal attention within each branch. Cross-branch pairs are masked out. For one full-attention layer, ignoring heads and batch, independent prompts admit

query-key pairs, whereas the tree admits

The exact difference is $(N-1)P(P+1)/2$: only repeated trunk self-attention disappears. Branch-to-trunk and branch-internal attention remain. With $P=3400$, three questions and $q_n=50$, token positions fall from 10,350 to 3,550, a 65.7% reduction. Permitted attention pairs fall from 17,858,925 to 6,295,525, a 64.7% reduction. These are operation-count derivations, not measured latency reductions.

Why split attention remains exact

Softmax attention cannot normally be computed by averaging two partial outputs. d1's implementation also returns the log-sum-exp normalizer from each part. Let $(o_a,\ell_a)$ be a branch query's attention over trunk keys and $(o_b,\ell_b)$ its attention over its own causal keys. With $m=\max(\ell_a,\ell_b)$, the complete result is

This is algebraically the same softmax as concatenating both key sets: $e^{\ell_a}$ and $e^{\ell_b}$ are their partition sums. Subtracting $m$ is the usual numerical-stability step. The equality applies to the defined attention mask and precision; the source notes that BF16 kernel shapes can still cause small numerical differences from separate runs.⁴

The optimization also has an implementation boundary. On Ampere-or-newer CUDA GPUs, or ROCm, with BF16/FP16 tensors, the custom code calls PyTorch's private flash-attention operators. CPU, unsupported GPU or float32 paths use explicit float32 products. For trunk attention they repeat K/V for GQA and split branch queries into chunks of 1,024; own-branch products are handled separately. A compatible public API or fused serving engine may have different performance and maintenance characteristics.

Typed answers from ordinary vocabulary logits

The last hidden state of every branch is projected by the tied 128,000-token language-model head. d1 does not decode from that distribution. Instead, its prompt code maps each allowed answer to one or more single-token forms, takes the maximum log-probability among an option's forms, then applies a softmax only across the allowed options:⁸

noul uses yes/no token aliases, choice uses generated option codes, and score uses digit tokens. This makes schema violations impossible at the API readout: output space is restricted before returning a result. It also means descriptions and option order can influence the prompt and selected aliases. The method is not an order-invariant classifier head.

The model card calls the returned answers calibrated. The public runner, however, defaults calibration=None; unless a caller supplies an object with apply, the reported “confidence” is only the restricted softmax above. It should not be interpreted as an empirically calibrated probability for a new production domain without measuring reliability on representative held-out data.

Production boundaries

The release requires trust_remote_code=True. Pinning only the model name allows repository code, tokenizer and weights to move independently over time. Download an inspected revision as a local snapshot, then load that path:

import torch
from huggingface_hub import snapshot_download
from transformers import AutoModel
 
REVISION = "051bcc464b01b9f92942b364d9586b0ef5912432"
snapshot = snapshot_download("LiquidAI/d1-3B", revision=REVISION)
model = AutoModel.from_pretrained(
    snapshot,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).to("cuda").eval()
 
result = model.system_one(state, questions)

The configured 32,768-token limit applies to the longest root-to-leaf path $P+\max q_n$, not the sum of all packed branches. Conversely, the runner's default token_budget=65536 controls how many real tokens are grouped into one batch; it does not raise the per-path context limit. Images are downscaled to at most 1,048,576 pixels before processing, and the processor pads images to its model-specific patch layout.⁹

Official GGUF files and a llama.cpp /v1/systemone path make local deployment practical. Quantization reduces stored weight size; it does not prove unchanged answer ranking or probability calibration. The official repository exposes a 1.67 GB Q4_K_M model file and a separate 0.86 GB BF16 multimodal projector, but no independent accuracy comparison for those quantized decisions was published at the checked revision.⁶

Liquid AI's latency table measured one request at a time. Three questions over one state took 20 ms versus 16 ms on Jetson AGX Thor, 35 versus 26 ms on AGX Orin, 41 versus 30 ms on Apple M5 Pro, and 21 versus 8 ms on RTX 4090. The shared trunk explains why cost need not triple, but the ratios vary widely; “1.3×” is not a universal property of tree inference.¹

Finally, the model uses the LFM Open License v1.0, not Apache 2.0 or MIT. Section 5 conditions commercial-use rights on a USD 10 million annual-revenue threshold and excludes above-threshold commercial use from that agreement. Organizations near or above that boundary should review the exact license and obtain appropriate permission before deployment.¹⁰

d1-3B's useful architectural idea is precise: factor a set of prompts into one shared causal prefix and independent suffixes, then preserve exact attention by merging partial softmax statistics. It saves repeated trunk work, not all question work. The benefit grows with the shared-state fraction and number of questions, while real throughput remains bounded by branch lengths, kernels, memory traffic, vision processing and batching.

Primary sources