Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head

Clef and Clef-flash are released multimodal decision models from Cloudflare, not research-only architecture proposals. Since 1 October 2026, both have been callable through Workers AI, and their BF16 weights, processor files, joint-head weights and reference Python implementation have been available under Apache 2.0. This article describes the published Clef revision 2f3de3d and Clef-flash revision 17f0b0a, checked on 2 October 2026.[^release][^clef-card][^flash-card]

They are not chat models. A request contains a state plus typed questions whose permitted answers are declared in advance. One forward pass returns one logit per permitted option; a separate softmax is applied within each question. There is no token-by-token answer generation and therefore no free-form output parser.[^clef-card][^implementation]

The two released variants

The layer and tensor values come from the released checkpoint configurations, not from rounded product names; the parameter totals are the repositories' safetensors metadata at the pinned revisions. Both Qwen backbones alternate three linear-attention layers with one full-attention layer and retain their vision encoder. This is related to, but not the same checkpoint as, Qwen3.8-Flash-Next. Clef does not add that model's n-gram memory.[^clef-card][^flash-card][^clef-config][^flash-config]

Input layout and the single backbone pass

The reference encoder turns a record into one causal sequence:

system instruction | state and optional media | schema fields and options | assistant suffix

For each field it records the token span of the question text and every option description. JSON states are serialized with sorted keys. Images and video frames are converted through the Qwen processor and inserted before the state. The schema is never silently shortened: if its fixed tokens exceed the configured limit, encoding fails. Otherwise, the state is truncated at the end so that the schema and suffix still fit.[^implementation]

Let $L$ be the valid input length, $h$ the backbone width, $F$ the number of fields and $O_f$ the option count of field $f$. After the Qwen prefill, the final hidden tensor is

$H \in \mathbb{R}^{L \times h}, \qquad h \in \{5120,4096\}.$

The release wrapper calls the text model with use_cache=False. It consumes the final hidden states immediately and does not retain a decode KV cache, because there is no autoregressive decode phase. This removes decode-time cache growth; it does not make the prefill free. Every fourth backbone layer still performs full causal attention, and the final $H$ plus a projected head memory both grow linearly with $L$.[^implementation][^clef-config][^flash-config]

Stage 1: option-specific evidence routing

The joint head first layer-normalizes $H$ and projects every token to a 1,024-wide memory:

$M = W_m\operatorname{LN}(H) \in \mathbb{R}^{L \times 1024}.$

For every question and option, the implementation mean-pools several spans. The resulting shapes for one record are:

Two EvidenceRoutingLayers update $X$. Each layer uses 16-head cross-attention from all option queries to $M$, so its head dimension is $1024/16=64$. A residual 1,024→4,096→1,024 GELU feed-forward block follows. Options do not self-attend in this stage; each independently retrieves evidence from the complete encoded record. Its attention work is proportional to $O\,L$, rather than $L^2$, but more declared options still increase compute and temporary attention storage.[^head-config][^implementation]

Stage 2: joint fields and cross-field interaction

The routed option vectors are split back into their fields. For each field, a dot product between its projected question vector and its options produces a softmax over that field's options. The weighted option summary is added to four terms:

1. the projected question vector;
2. the option summary;
3. a projection of the final sequence token;
4. a learned embedding for the noul, choice, or score type.

This forms $F$ field vectors of width 1,024. Four standard pre-norm TransformerDecoderLayers then apply unmasked self-attention across the fields, cross-attention back to the full memory $M$, and a 4,096-wide feed-forward block. The important distinction is concrete: stage 1 routes state evidence into individual options; stage 2 lets fields interact and revisit the state. Field self-attention costs $O(F^2)$, while the four memory cross-attentions cost $O(F\,L)$.[^implementation]

Schema-bound scoring and the lexical prior

Each final field vector is scored only against its own routed options. The implementation combines two paths: