Nemotron 3.5 Embed: adding images without replacing the text index

AI disclosure: This article was prepared with generative AI assistance. Editorial responsibility: Florian Zimmermeister (flozi00). Legal notice and contact.

NVIDIA released the open-weight Nemotron 3.5 Embed family on October 20, 2026: 8B BF16, 1B BF16 and 1B NVFP4 checkpoints, plus an 8B multimodal reranker. These are downloadable models with documented Transformers, Sentence Transformers, vLLM and NIM paths, not an API-only preview. The governing model license is OpenMDW-1.1.¹²

The technically useful feature is a migration property: for each size, the language component comes from the corresponding text-only Nemotron 3 Embed model and remains frozen while the vision encoder and multimodal projector learn. NVIDIA therefore says text-only embeddings remain unchanged and an existing text index can be reused when adding image documents. This is a narrower and more valuable claim than “all models share one space.” The release does not promise that arbitrary 1B and 8B vectors, different dimensions, prefixes or preprocessing pipelines may be mixed.¹³

This guide follows the public artifacts as of October 22, 2026. Equations below explain the mechanism under stated assumptions. They do not reconstruct undisclosed loss weights or report an independent quality or throughput benchmark.

What is actually released

Queries are text-only. A document can be text, a page image, or image plus text. The embedding models are bi-encoders: query and document are encoded independently, so document vectors can be computed once and searched repeatedly. The reranker is a cross-encoder: it jointly processes one query–document pair and emits a scalar, so it belongs after first-stage retrieval rather than over the full corpus.²⁴

The documented input ceiling is 32,768 combined text-and-image tokens. The default image longest_edge is 1,120 pixels and can be raised to 1,540. Those are processor limits, not promises that every document preserves small print or that token count is independent of aspect ratio.²

The data path: many token states become one vector

The model card's reference implementation applies the saved query: or passage: template, runs the multimodal Transformer, mean-pools non-padding token states and L2-normalizes the result. If $H\in\mathbb{R}^{T\times d}$ is the final token-state matrix and $m_i\in\{0,1\}$ is the attention mask, the published path is:

Because $e_q$ and $e_d$ have unit norm, the dot product is cosine similarity. This is single-vector dense retrieval: storage is one vector per indexed unit, unlike the token-matrix and MaxSim scorer in pplx-embed-v2. Mean pooling is cheap, but it compresses all page regions into one point; it cannot expose which chart cell won a token-level match.²

The model card names a Pixtral vision encoder and a multimodal projection. It does not publish enough shape detail to derive every intermediate visual tensor safely. What is established is the boundary: the trained visual path supplies token representations that are pooled into the same kind of normalized vector as text, and only the visual encoder/projector were adapted for the 8B model.³

Why freezing the text path makes index reuse possible

Let $F_0(x)$ denote the old text-only encoder including the retrieval prefix, tokenizer, pooling and normalization, and let $F_{3.5}(x)$ be the new model's text-only path. If all text-path parameters and preprocessing are unchanged, freezing gives the functional identity

That is stronger than merely keeping the same dimensionality: two unrelated encoders can both emit 2,048 numbers while using rotated or otherwise incompatible coordinates. Here the old text mapping is preserved and a trainable visual mapping $G_\phi(I,t)$ is fitted into that coordinate system. A schematic contrastive objective for a batch of paired queries and documents is

The release says the 8B visual path was trained contrastively, but does not disclose the exact batches, temperature or all loss terms; the equation is explanatory. Its geometry is the important part: relevant visual documents are pulled toward their text queries, and batch negatives are pushed lower, while the text coordinate system itself stays fixed.¹

Operationally, “reuse the index” applies to the corresponding predecessor and size with identical prefixes, tokenizer, pooling, normalization, dimension and document preprocessing. A byte-for-byte regression test on representative text is still appropriate: runtime kernels and floating-point order may introduce small numerical differences even when the mathematical mapping is unchanged. Changing a prefix from passage: to query: changes the input and voids the identity.

Why cosine plus MSE helps the 1B student

The 1B multimodal model is distilled from the 8B teacher. NVIDIA states that training combines cosine-distance and mean-squared-error losses over multilingual text and image retrieval data.²¹ There is an essential shape constraint: the cards describe native outputs of 2,048 and 4,096 coordinates, so subtracting those raw vectors is undefined. The release does not specify the dimension-matching operation or the exact representations to which its losses apply.

To explain the loss without inventing that missing bridge, let $u_s,u_t\in\mathbb{R}^{r}$ denote already matched representations in a common $r$-dimensional space. Only under this stated assumption is the following illustrative objective well-typed:

The cosine term aligns direction. If its inputs are unnormalized, MSE additionally penalizes scale differences. If both inputs are unit-normalized, however, $\lVert u_s-u_t\rVert_2^2=2(1-u_s^\top u_t)$: MSE and cosine distance then differ only by a constant factor. The public loss names alone do not establish which case the released training uses.

For unit-normalized teacher/student query and document representations in that same common space, score drift has a simple bound:

This follows by adding and subtracting $q_t^\top d_s$ and applying Cauchy–Schwarz. If every candidate score differs from the teacher by at most $\varepsilon$, a teacher margin greater than $2\varepsilon$ preserves that pair's order. This conditional bound does not apply directly to the differently sized native API outputs, establish an undisclosed projection, or authorize mixing 1B and 8B vectors. The release supplies neither that bridge nor per-vector error bounds. Same-size predecessor index reuse is supported by freezing, independently of this illustrative distillation analysis.

Dynamic dimensions: smaller indexes, a different contract

The 1B card documents a 2,048-dimensional native vector and truncation from the end, followed by mandatory re-normalization. NIM release 2.4 accepts any requested dimension from 128 through 2,048. For $D$ retained coordinates and $b$ bytes per component, raw vector storage is simply $NDb$.²⁵

These are raw payloads, excluding HNSW links, metadata, replicas and allocator overhead. All vectors compared in one index must use the same retained dimension and normalization. The release does not provide a universal recall curve for 128, 512 and 2,048 dimensions; choose the dimension on a labeled validation set, then rebuild or version the collection rather than silently mixing shapes.

There is also an official documentation mismatch worth treating as a deployment stop sign: the 8B model card says 4,096 native dimensions, while the current NIM support matrix lists 2,048 for its 8B profile. That may describe a serving projection or truncated profile, but the cited pages do not resolve it. Inspect the actual response shape and pin the runtime/model revision before allocating an index. Do not infer 1B/8B interchangeability from the NIM table.³⁵

NVFP4 quantizes model execution, not automatically the index

The 1B NVFP4 checkpoint applies NVFP4 weight-and-activation quantization to eligible linear layers in both language and vision components, then uses quantization-aware distillation mainly to recover long-input accuracy. It targets vLLM 0.32.0 and NVFP4-capable NVIDIA Blackwell hardware. BF16 is the portable checkpoint for documented Transformers and Sentence Transformers use.⁶

Keep three precision choices separate:

• checkpoint precision controls stored model weights;
• kernel activation precision controls eligible matrix operations;
• embedding output type controls vector-database payloads.

NIM exposes float, int8 and uint8 embedding outputs for this family. Base64 is only an encoding format; it does not change vector precision. A four-bit checkpoint can therefore still emit float32 vectors, and requesting int8 output does not make unsupported hardware execute NVFP4.⁵

NIM 2.4 also distinguishes a checkpoint label from execution precision: its 1B-BF16 checkpoint runs in FP16, and an explicit NIM_ENGINE_PRECISION=bf16 override is not implemented. Its deployment sets the 32,768-token limit explicitly; without NIM_PIPELINE_MAX_SEQ_LEN, the server fallback is 512. Record resolved precision and sequence limits rather than treating a model-card maximum as the service default.⁵

NVIDIA reports, on GB200, 101.3 documents/s for 1B BF16 and 128.8 documents/s for 1B NVFP4, with 99.85% of its reported ViDoRe-v3 score retained. These are vendor measurements; the prose does not fully specify batch composition, image sizes or concurrency. Treat them as one documented operating point, not a capacity forecast.¹

Production checklist

1. Pin the exact model, processor and runtime versions; the cards document separate Transformers 5.20.0 and vLLM 0.32.0 environments.
2. Preserve query: and passage: roles. A dimension match without the same template is not compatibility.
3. Before reusing a Nemotron 3 text index, compare old/new text vectors and top-k results on representative documents. Reuse only the corresponding model size and output dimension.
4. Record image resize settings, combined token counts and truncation. The 32,768-token limit includes image tokens.
5. Validate text-only, image-only and image-plus-text subsets separately. A single aggregate score can hide a modality regression.
6. If using reduced dimensions or int8 outputs, measure recall after each change; do not combine both changes and attribute the result to one.
7. Add the 8B cross-encoder only to a bounded candidate set. It improves ranking at a cost proportional to the number and length of pairs.
8. Keep the source page or chunk beside every vector. Similarity returns a candidate, not evidence provenance or a generated answer.

The architecture's practical win is controlled extension: preserve a proven text coordinate system, train images into it, and distill a smaller serving model against the larger teacher. Its boundary is equally important: compatibility is a versioned numerical contract, not a property implied by a shared model-family name.

For index memory, ANN overhead and latency floors, continue with the RAG retrieval math guide. For an architecture that intentionally uses a small query encoder with a larger document encoder, see pplx-embed-v2.

Primary sources