The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT

Self-hosting a serving engine in 2026 means running a stack that changes underneath you weekly. In the first days of September alone, all three major engines shipped releases that can break a production deployment: SGLang 0.5.19 on September 5, vLLM 0.29.0 on September 9, and NVIDIA's TensorRT-LLM v1.3.0rc26 — also September 9, part of a release-candidate line shipping roughly every one to two weeks since late June ^{1 2 3}.

This is not a complaint about velocity — the releases are full of genuine improvements. It is a cost accounting: pins to maintain, migrations to test, regressions that never announce themselves in a changelog. This guide verifies the three releases at their primary sources, computes what an upgrade gone wrong costs, and gives a pinning discipline that survives the cadence. Engine selection is covered in the vLLM vs SGLang comparison; this piece is about surviving whichever one you picked.

The three releases at a glance

vLLM 0.29.0: a new default runner, and the old one is on a timer

vLLM 0.29.0 (published September 9, 2026) makes Model Runner V2 the default for all models, and announces the corollary in the same breath: Model Runner V1 is deprecated, with removal targeted for v0.32 ².

The operationally dangerous part, in substance from the release notes: some features are not yet supported in MRV2, and vLLM silently falls back to MRV1 when they are configured. The named gaps: sequence parallelism, dual-batch overlap, elastic expert parallelism, custom logits processors, and certain speculative decoding methods ². Config and model do not change; the code path your traffic takes does — decided at startup from your flags, the only symptom a throughput number drifting down. The textbook case for benchmarking your config, not the release blog's.

A second hard boundary: deprecated architecture entries were removed, including Arctic, Chameleon, Cheers, Fairseq2Llama, FireRedLID, GritLM, HCXVision and MPT. Checkpoints that still declare one of these unsupported architectures fail with a named error. The release notes also list the RWForCausalLM and StableLMEpochForCausalLM aliases and PrithviGeoSpatialMAE (superseded by Terratorch); the notes call the set “ten” while enumerating eleven names, so a migration should check its actual checkpoint architecture rather than rely on that count. The PyAV video-decoder backend went too, for OpenCV or Torchcodec ². And FlashInfer all-reduce is now on by default for TP CUDA groups ² — an "engine upgrade" is a bet that a dozen flags still compose the same way. The MRV1-to-MRV2 transition is a behavioral change wearing a version change's clothes: with v0.32 coming, if you use sequence parallelism or dual-batch overlap in production, you are planning a migration, not an upgrade.

SGLang 0.5.19: beam search arrives, and the default cache changes

SGLang v0.5.19 (published September 5, 2026) carries 786 merged pull requests from 214 contributors — a substantial release to evaluate ¹.

The headline is beam search: pass beam_width in a request and get the n best sequences instead of one sample ¹. The caveat sits in the same sentence: it does not yet mix with speculative decoding, prefill/decode disaggregation, DP attention, or HiCache ¹. If your stack uses any of those four, the headline feature is unavailable to you — "the release has feature X" and "feature X ships to my deployment" are different claims.

The churn is in the breaking-changes section. The unified radix tree is now the default cache for every configuration, its opt-in env var deprecated — a swap of the KV cache's core data structure (background: the site's KV-cache deep dive) as a side effect of a minor bump. Further: Spark3 is renamed Spark2.5 across config, model classes, and the tool-call parser; FlashInfer 0.6.18 is a hard requirement, no fallback on two named paths; ServerArgs no longer auto-resolves (call resolve_once() if you build your own record); requests are capped at 32 stop strings of 256 bytes, HTTP 400 beyond that ¹. Since that release, v0.5.20 (September 18) has already landed: 713 PRs from 237 contributors, and its breaking changes bite harder than 0.5.19's — CUDA 12 wheels and images are retired (0.5.19 is the last CUDA 12 release; pin it if CUDA 13 is not an option yet), prefill context-parallelism v1 is removed, and /v1/responses persistence is opt-in now. Those September releases require explicit compatibility checks.

TensorRT-LLM: "without TensorRT" — but not where you were told

Here verification pays off. The circulating claim is that "TensorRT-LLM 1.2 removes the TensorRT backend." The primary sources say: wrong on the version. Stable v1.2.0 (March 12, 2026) still shipped both backends; its breaking changes concern sampling and container baselines ⁴.

The removal lives in the v1.3.0 RC line (~1 RC per 10 days since July) ^{5 3}: rc21 (July 15) removed the Python modules and tests for the legacy backend; rc22 removed the C++ modules (rc23 cleaned up the leftover relics and docs). In the current line: LLM(backend="tensorrt") raises a ValueError; the trtllm-build / trtllm-refit / trtllm-prune CLIs, the per-model convert_checkpoint.py scripts, and the --backend tensorrt CLI choice are all gone; PyTorch is the sole execution backend — HuggingFace checkpoints load directly, no engine-build step ^{6 7}. The product retains its name while this legacy execution backend is removed; migration guides and runbooks must specify the release line.

The second structural change is the KV Cache Manager V2, the notes' recommended architecture — new models default to V2, existing models migrate gradually, V1 will be deprecated ⁸. Storage-side it is no cosmetic rewrite: KVCM2 supports a hot/cold tier split where evicted pages are encoded into a fixed-size opaque cold-page blob — one I/O operation per cold page instead of per hot pool, optional compression (NVFP4 cold-page quantization), hot-to-host and hot-to-disk migrations ⁹ — a second subsystem whose defaults shifted mid-year.

The bind for self-hosters: pin v1.2.0 and you run a March release while the RC line sits two subsystem rewrites ahead; chase v1.3.0rc28 (September 23) and you run release candidates in production, whose known-issues list includes disaggregated-serving startup failures and a TinyLlama batch-isolation bug ¹⁰. The self-hoster must weigh the older stable release against the risks and fixes of the RC line.

The churn-tax math, computed

Take a modest setup: one endpoint at 99.9% availability SLO. A 30-day month has 43,200 minutes, so the error budget is 0.1% of that: 43.2 minutes per month — the entire slack a botched upgrade may consume.

An upgrade gone wrong. 10 minutes to detect, 15 to roll back (the image is pinned, so this is restart-with-old-digest, not "find a working version"), 8 minutes of warmup — an empty KV cache means cold prefixes and zero prefix-cache hits. Total: 33 minutes, or 76.4% of the monthly error budget. Two incidents in a month cost 66 minutes, 152.8% of the budget: the SLO is mathematically blown, at an achieved 99.847%, before any outage unrelated to maintenance.

The silent-regression case. That 33-minute scenario is the loud failure. The expensive one is quiet: vLLM's MRV1 fallback means a config using dual-batch overlap can silently change execution paths. A 20% throughput regression on a 4-GPU node unnoticed for five days: 4 GPUs × 120 hours × 20% = 96 GPU-hours of lost throughput capacity, equivalent to $240 if valued at $2.50 per GPU-hour — and the real damage is capacity planning: peak-load headroom quietly shrank by a fifth.

The absorption cost. Suppose one team evaluates 12 vLLM releases, 26 SGLang releases and 12 TensorRT-LLM checkpoints per year: 50 candidate releases. At an assumed two engineer-days per full A/B, testing every one would require 100 engineer-days, or 0.45 of a 220-day engineer-year. These are planning assumptions, not a measured 2026 release count; a team can triage and skip releases.

What you must actually pin

The naive pin is "engine version + weights hash." It is insufficient: a weights hash pins nothing without a runtime hash. The same checkpoint under vLLM 0.29's MRV2 and 0.28's MRV1 takes different code paths with different feature support; the same model under TensorRT-LLM's KVCM V1 and V2 has different tiering and eviction behavior. The full pin surface for a reproducible deployment:

1. Image digest — the immutable sha256: reference, not the tag.
2. Engine version — redundant given the digest, but useful in incident reports.
3. Flags and config — env vars like VLLM_ATTENTION_BACKEND change effective behavior per engine version (MRV1 fallback!), so config alone pins nothing.
4. Weights hash — commit the checksum; a floating revision re-quantized differently is not the artifact you benchmarked.
5. Kernel/dependency pins — SGLang 0.5.19 requires FlashInfer 0.6.18 with no fallback on named paths ¹.
6. Eval baselines — golden outputs you diff against, so "same weights, new engine" is tested for behavior, not just throughput.

Six artifacts, not two — every release above touches a subset of them.

Pin the digest, not the tag. Tags are mutable: vllm/vllm-openai:v0.29.0 can be re-pushed; latest floats mid-deploy. Pin the immutable form:

image: vllm/vllm-openai@sha256:<digest>   # what you deploy

Retrieve it once (docker inspect --format '{{index .RepoDigests 0}}' or crane digest), store it in the manifest, and treat any digest change like a version bump: changelog, canary, rollback plan.

Upgrade or freeze? A decision framework

The right posture is asymmetric: fast for security and model support, slow for performance.

Prioritize a security fix according to the vulnerability’s severity and your exposure, or a model-support update that you need. Test the candidate and deploy it on a risk-appropriate schedule.

Freeze deliberately when in production serving and the notes contain neither. Your engine choice was made with benchmarks true for the pinned version; nothing upstream improves your deployed reality without re-running them on your hardware — the vLLM vs SGLang guide makes the same point at selection time. Choose the review cadence from your support and security requirements.

A/B a runtime with the same weights — the core skill this cadence demands, with MRV2 as the worked example:

1. Bring up the candidate on the same node class, same weights digest, same flags.
2. Replay recorded traffic or a fixed benchmark with pinned seeds — the release notes were validated on someone else's traces.
3. Diff latency distribution (p50/p99), throughput at real concurrency, and output equivalence — a scheduler or sampling change can alter generations without any error anywhere.
4. Check which runner actually ran: for vLLM 0.29.0, confirm in the logs whether MRV1 silently engaged ². If it did, your A/B measured the old runner — and says nothing about the path you run after v0.32.

Promote only on all passing, keep the old digest warm for instant rollback, and budget the 33-minute recovery into the window.

The critical view

Release-blog benchmarks are cherry-picked kernels, not your end-to-end. vLLM's notes quote kernel speedups (6.6–7.6× on a Mamba metadata kernel, 12.9–25.2% on a GEMM path); SGLang quotes up to 1.52× decode throughput for a specific AMD kernel on specific hardware ^{2 1}. Honest about their scope — but single-kernel measurements on vendor-adjacent hardware, while your bill is end-to-end tokens per second on your fleet. The only benchmark that binds is the one you run.

"Default to vLLM" is now a moving-target statement. "vLLM" in the sense of what your flags do changed on September 9, when MRV2 became default with per-feature fallback rules, and changes again by v0.32, when MRV1 is removed. Any guide — including this site's — is a statement about a version window.

Cadence-as-strategy, honestly read. The engine war benefits users in aggregate: features reach everyone in weeks, not quarters. But aggregate benefit and individual cost are different ledgers. Capturing the benefit requires absorbing the upgrades, and the illustrative 50-candidate annual queue still needs triage. Cloud providers amortize absorption across thousands of tenants; a self-hoster with pinning discipline pays it deliberately, in windows they choose.

These three September releases show why version-specific compatibility checks matter. The crisis is running this stack as if the 2024 cadence still applied: unpinned images, "engine version + weights" as if that pins anything, benchmarking as a one-time act. The stack that survives pins the digest, records its six artifacts, and treats every upgrade as an A/B with a rollback path. Choose your engine with the architecture comparison; if you arrived via llama.cpp/Ollama, the discipline here is the tax on leaving ollama serve. Run the self-hosting math before committing GPU dollars.