Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing

Bria published Fibo 2.0 Dev for image generation and editing and Fibo 2.0 Edit Turbo for distilled four-step editing. Hugging Face merged the production Diffusers integration on 20 October 2026. This article audits that merged implementation at commit de988b7ee77c393a63865d265d19e267ef9c7bf5; it does not infer the architecture from product claims.¹²³

The useful mechanism is a two-route conditioning system. Qwen3-VL features are compressed into 256–512 gist tokens that participate in every main DiT block, while five selected blocks also cross-attend to the full Qwen feature sequence. During editing, each source image enters a second time as clean VAE latents in the same self-attention stream as the noisy target. Compression therefore bounds one expensive path without making the complete text sequence unavailable.⁴⁵

AI disclosure: This article was prepared with generative AI assistance. Editorial responsibility: Florian Zimmermeister (flozi00). Legal notice and contact.

The released pipeline, not just the 8.1B label

The Diffusers documentation calls the denoiser an 8.1-billion-parameter flow-matching transformer. That number does not describe the complete resident pipeline: inference also loads Qwen3-VL-4B-Instruct as the multimodal encoder and a FLUX.2 VAE. The checkpoint repositories are gated; the model pages and merged implementation were inspectable, but this review did not download weight payloads or inspect learned gate values.¹⁶

A 2×2 patch of the 32-channel latent contains 128 scalars and is projected to width 3,584. Because the VAE latent grid is one eighth of the pixel resolution, one DiT token covers a 16×16 pixel region. A 1,024×1,024 target therefore produces $64\times64=4{,}096$ target tokens. This is a shape derivation, not an image-quality or throughput measurement.⁴⁵

The 128 dimensions of each attention head are split across three rotary axes. Gist tokens advance on the first axis. The noisy target always occupies plane 513, because the code reserves the maximum 512 gist positions rather than the prompt's actual gist count. Source image $k$ occupies plane $513+k$, with its own row and column grid. This keeps image geometry distinct while letting all tokens share ordinary non-causal self-attention.⁴

Qwen3-VL is read six times in different bundles

The pipeline runs Qwen3-VL with hidden-state output enabled. It applies per-token RMS normalization in FP32 with epsilon $10^{-6}$ and casts back to the encoder dtype. It then concatenates three selected states, producing a 7,680-wide bundle per token. The numbers below are indices into the returned hidden_states tuple, whose index 0 is the embedding output; they are not zero-based decoder-block numbers. For example, tuple index 9 follows decoder block 8.⁵⁷

This is not six executions of Qwen3-VL. One encoder pass exposes multiple hidden layers; the pipeline rearranges them into a tensor of shape [B, 6, L, 7680]. The first bundle feeds the Perceiver. Each of the remaining five is consumed once by its assigned DiT block.⁵

At an injection block, queries come from the 3,584-wide DiT stream while keys and values are projected from the 7,680-wide Qwen bundle. The residual is multiplied by $anh(g_\ell)$, where $g_\ell$ is a learned scalar initialized to zero in the implementation class. Initialization describes the code path, not the released checkpoint's learned scalar value, which was not audited.⁴

Dynamic gist compression and its mathematics

Let $L$ be the number of real Qwen tokens after truncation to at most 4,096. Fibo maps text lengths from 256 to 2,048 linearly onto 256 to 512 gist queries, rounds upward in steps of 64, and clamps outside that interval. The exact implementation is:⁴

Each of the two Perceiver layers first cross-attends from learned queries [B, G, 3584] to the Qwen bundle [B, L, 7680], then self-attends among the $G$ queries and applies a feed-forward block. Its leading attention work therefore scales with $GL+G^2$, not $L^2$. The resulting gist vectors enter the common DiT stream.⁴

The compression is lossy: 512 vectors cannot generally preserve every degree of freedom in 4,096 source vectors. But it is not the only text route. Five main blocks still cross-attend to all $L$ Qwen positions. The architecture trades continuous full-sequence access in all 44 self-attention blocks for a bounded summary everywhere plus five full-resolution refresh points. Whether this improves image quality is an empirical question; the source establishes the dataflow, not the effect size.

Why the bounded stream matters

For a target with $N$ tokens, context images with $C_k$ tokens and $G$ gist tokens, each main block self-attends over:

With a long prompt ($G=512$) and 1,024×1,024 images ($N=C_k=4{,}096$), the sequence and quadratic score-pair count grow as follows. IO-aware kernels can avoid materializing the full score matrix, but they do not remove the pairwise dot-product work.⁴

If all 2,048 Qwen positions joined the stream instead of 512 gist tokens, the no-source sequence would be 6,144 and its score-pair term would be 1.78× larger. That comparison isolates only the main-stream text contribution. Qwen encoding still costs work, the two Perceiver layers still cross-attend over $L$, and five injection blocks still perform full-text cross-attention. Gist compression does not make the complete pipeline independent of prompt length.

The table also exposes the editing limit: supporting five source images is a functional maximum, not a claim that such a request is cheap. At equal one-megapixel sizes, five clean contexts increase the main self-attention pair term by almost 30× before accounting for the two noise-refiner blocks. Actual memory depends strongly on the selected attention backend, precision and activation strategy.

Editing sends each source image through two paths

A source image is first resized for Qwen3-VL, with each side capped at 768 pixels. The encoder can use those visual tokens to interpret objects, text and the edit instruction. Separately, the FLUX.2 VAE encodes the cropped image into a 32-channel latent tensor. The DiT patchifies that tensor with the same projection as the noisy target and appends it as clean context tokens.⁵⁸

The target positions receive the current flow timestep. Gist and context positions receive the embedding for $t=0$. Per-token AdaLN scales and residual gates select between the noisy and clean timestep embeddings. All positions interact through self-attention, but the final layer slices out and predicts only the target positions; clean context positions are never decoded as output.⁴

This dual representation has a concrete division of labor:

• The Qwen route supplies semantic, instruction-aware features at text-token resolution.
• The VAE route supplies spatially arranged image latents to every self-attention block.
• A source image can therefore influence generation through both semantic features and direct latent context.

That dataflow can preserve information that either path alone might omit, but the code does not prove a quality advantage. It also duplicates work: source images are processed by both Qwen3-VL and the VAE, then their latent tokens enlarge every DiT attention layer.

A mask is preprocessing, not a hard output constraint. The implementation thresholds it at 128 and replaces the selected source pixels with flat RGB $(128,128,128)$ before resizing and VAE/Qwen encoding. There is no final pixel compositor that copies unmasked source pixels back. “Masked edit” therefore means conditioned regeneration from a greyed source, not mathematically guaranteed invariance outside the mask.⁸⁶

Dev versus Edit Turbo: exact call count, not a speed benchmark

The Dev defaults are 30 denoising steps with guidance scale 5. Any guidance value above one triggers classifier-free guidance: every step runs the transformer once for the positive condition and once for the negative condition. The default Dev path therefore executes 60 transformer calls.⁵⁶

Edit Turbo is documented for four editing steps with guidance scale 1, which disables the negative pass. That path executes four transformer calls, a 15× reduction in denoiser-call count. It is not automatically a 15× latency or cost improvement: Qwen3-VL and VAE work, model loading, attention sequence length, kernel efficiency, offloading and scheduling remain. The Turbo card also scopes the checkpoint to editing; text-to-image remains the Dev checkpoint's supported path.²⁶

Production boundaries

• Variable-length batches need a mask-capable backend. Padded gist tokens are moved to the end because variable-length FlashAttention backends retain the first valid keys. A non-mask backend such as _flash_3_hub is only suitable for a single prompt or equal lengths. Diffusers notes that BF16 padded and unpadded kernels can differ in fine details.⁶³
• Full-graph compilation breaks on dynamic gist counts. The implementation converts per-prompt lengths with .tolist(). The integration records this as a torch.compile(transformer) graph break; regional compilation of repeated blocks remains available.³
• Text-image conditioning changes during generation. When a caption requests rendered text, generation initially omits the auxiliary text images and enables them at noise level $t\le0.9$. Editing uses them on every step. This is a deliberate conditioning schedule, not an invariant prompt embedding.⁵
• The model pages are gated. Access requires accepting the repository terms. The Hub metadata labels the license bria-fibo; the cards describe non-commercial model access and a separate commercial agreement. This article makes no legal-compatibility claim.¹²
• No local performance result is claimed. The Diffusers PR reports reference parity tests on A100 and H100. This review checked source and arithmetic only; it did not run the weights, measure VRAM, throughput, latency or output quality.³

What is actually new

Fibo 2.0 does not replace attention with a new primitive. Its architectural contribution is the composition of familiar mechanisms: a multimodal encoder, length-adaptive Perceiver queries, a single-stream flow transformer, sparse full-text refresh points and clean image latents distinguished by timestep and 3D RoPE coordinates.

The important production trade is visible in the equations. Gist tokens cap text's contribution to all-block quadratic self-attention, while five full-sequence injections keep a high-resolution route. Clean contexts avoid a separate image cross-attention subsystem, but make editing cost grow quadratically with the combined target and source-token count. For comparison, the Qwen-Image 2.1 guide uses block-causal attention and a reusable prefix cache; Fibo 2.0 instead keeps target and source latents in one bidirectional stream at every step.