The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT

Self-hosting a serving engine in 2026 means running a stack that changes underneath you weekly. In the first days of September alone, all three major engines shipped releases that can break a production deployment: SGLang 0.5.19 on September 5, vLLM 0.29.0 on September 9, and NVIDIA's TensorRT-LLM v1.3.0rc26 — also September 9, part of a release-candidate line shipping roughly every one to two weeks since late June ^{1 2 3}.

This is not a complaint about velocity — the releases are full of genuine improvements. It is a cost accounting: pins to maintain, migrations to test, regressions that never announce themselves in a changelog. This guide verifies the three releases at their primary sources, computes what an upgrade gone wrong costs, and gives a pinning discipline that survives the cadence. Engine selection is covered in the vLLM vs SGLang comparison; this piece is about surviving whichever one you picked.

The three releases at a glance

vLLM 0.29.0: a new default runner, and the old one is on a timer

vLLM 0.29.0 (published September 9, 2026) makes Model Runner V2 the default for all models, and announces the corollary in the same breath: Model Runner V1 is deprecated, with removal targeted for v0.32 ².

The operationally dangerous part, in substance from the release notes: some features are not yet supported in MRV2, and vLLM silently falls back to MRV1 when they are configured. The named gaps: sequence parallelism, dual-batch overlap, elastic expert parallelism, custom logits processors, and certain speculative decoding methods ². Config and model do not change; the code path your traffic takes does — decided at startup from your flags, the only symptom a throughput number drifting down. The textbook case for benchmarking your config, not the release blog's.

A second hard boundary: ten deprecated architectures were removed — Arctic, Chameleon, Cheers, Fairseq2Llama, FireRedLID, GritLM, HCXVision, MPT — checkpoints for those fail with a named error, Model architecture X was supported in vLLM until v0.28.0, and is not supported anymore. The other three of the ten were registry aliases whose native implementations stay untouched: RW loads as Falcon, StableLMEpoch as StableLM, and PrithviGeoSpatialMAE is superseded by the generic Terratorch architecture its checkpoints already declare — they keep loading. The PyAV video-decoder backend went too, for OpenCV or Torchcodec ². And FlashInfer all-reduce is now on by default for TP CUDA groups ² — an "engine upgrade" is a bet that a dozen flags still compose the same way. The MRV1-to-MRV2 transition is a behavioral change wearing a version change's clothes: with v0.32 coming, if you use sequence parallelism or dual-batch overlap in production, you are planning a migration, not an upgrade.

SGLang 0.5.19: beam search arrives, and so does a mandatory cache swap

SGLang v0.5.19 (published September 5, 2026) carries 786 merged pull requests from 214 contributors — a week of integration effort in one tag you are expected to consume ¹.

The headline is beam search: pass beam_width in a request and get the n best sequences instead of one sample ¹. The caveat sits in the same sentence: it does not yet mix with speculative decoding, prefill/decode disaggregation, DP attention, or HiCache ¹. If your stack uses any of those four, the headline feature is unavailable to you — "the release has feature X" and "feature X ships to my deployment" are different claims.

The churn is in the breaking-changes section. The unified radix tree is now the default cache for every configuration, its opt-in env var deprecated — a swap of the KV cache's core data structure (background: the site's KV-cache deep dive) as a side effect of a minor bump. Further: Spark3 is renamed Spark2.5 across config, model classes, and the tool-call parser; FlashInfer 0.6.18 is a hard requirement, no fallback on two named paths; ServerArgs no longer auto-resolves (call resolve_once() if you build your own record); requests are capped at 32 stop strings of 256 bytes, HTTP 400 beyond that ¹. Since that release, v0.5.20 (September 18) has already landed: 713 PRs from 237 contributors, and its breaking changes bite harder than 0.5.19's — CUDA 12 wheels and images are retired (0.5.19 is the last CUDA 12 release; pin it if CUDA 13 is not an option yet), prefill context-parallelism v1 is removed, and /v1/responses persistence is opt-in now. Half a year of churn in a single week.

TensorRT-LLM: "without TensorRT" — but not where you were told

Here verification pays off. The circulating claim is that "TensorRT-LLM 1.2 removes the TensorRT backend." The primary sources say: wrong on the version. Stable v1.2.0 (March 12, 2026) still shipped both backends; its breaking changes concern sampling and container baselines ⁴.

The removal lives in the v1.3.0 RC line (~1 RC per 10 days since July) ^{5 3}: rc21 (July 15) removed the Python modules and tests for the legacy backend; rc22 removed the C++ modules (rc23 cleaned up the leftover relics and docs). In the current line: LLM(backend="tensorrt") raises a ValueError; the trtllm-build / trtllm-refit / trtllm-prune CLIs, the per-model convert_checkpoint.py scripts, and the --backend tensorrt CLI choice are all gone; PyTorch is the sole execution backend — HuggingFace checkpoints load directly, no engine-build step ^{6 7}. A product named TensorRT-LLM that no longer contains TensorRT is the purest expression of the churn tax: runbook name and software have diverged.

The second structural change is the KV Cache Manager V2, the notes' recommended architecture — new models default to V2, existing models migrate gradually, V1 will be deprecated ⁸. Storage-side it is no cosmetic rewrite: KVCM2 supports a hot/cold tier split where evicted pages are encoded into a fixed-size opaque cold-page blob — one I/O operation per cold page instead of per hot pool, optional compression (NVFP4 cold-page quantization), hot-to-host and hot-to-disk migrations ⁹ — a second subsystem whose defaults shifted mid-year.

The bind for self-hosters: pin v1.2.0 and you run a March release while the RC line sits two subsystem rewrites ahead; chase v1.3.0rc28 (September 23) and you run release candidates in production, whose known-issues list includes disaggregated-serving startup failures and a TinyLlama batch-isolation bug ¹⁰. Most users consume TensorRT-LLM through managed endpoints; the self-hoster pays the RC tax directly.

The churn-tax math, computed

Take a modest setup: one endpoint at 99.9% availability SLO. A 30-day month has 43,200 minutes, so the error budget is 0.1% of that: 43.2 minutes per month — the entire slack a botched upgrade may consume.

An upgrade gone wrong. 10 minutes to detect, 15 to roll back (the image is pinned, so this is restart-with-old-digest, not "find a working version"), 8 minutes of warmup — an empty KV cache means cold prefixes and zero prefix-cache hits. Total: 33 minutes, or 76.4% of the monthly error budget. Two incidents in a month cost 66 minutes, 152.8% of the budget: the SLO is mathematically blown, at an achieved 99.847%, before any outage unrelated to maintenance.

The silent-regression case. That 33-minute scenario is the loud failure. The expensive one is quiet: vLLM's MRV1 fallback means a config using dual-batch overlap can silently change execution paths. A 20% throughput regression on a 4-GPU node unnoticed for five days: 4 GPUs × 120 hours × 20% = 96 wasted GPU-hours, $240 at $2.50/GPU-hour — and the real damage is capacity planning: peak-load headroom quietly shrank by a fifth.

The absorption cost. vLLM ships monthly minors, SGLang roughly biweekly, TensorRT-LLM a usable checkpoint roughly monthly — on the order of 74 release events per year worth evaluating. At two engineer-days per serious A/B, that is 148 engineer-days, or 0.67 of a full-time engineer-year (at 220 workdays) spent purely absorbing upstream churn. That fraction is the churn tax, in the only unit that matters: your team's attention.

What you must actually pin

The naive pin is "engine version + weights hash." It is insufficient: a weights hash pins nothing without a runtime hash. The same checkpoint under vLLM 0.29's MRV2 and 0.28's MRV1 takes different code paths with different feature support; the same model under TensorRT-LLM's KVCM V1 and V2 has different tiering and eviction behavior. The full pin surface for a reproducible deployment:

1. Image digest — the immutable sha256: reference, not the tag.
2. Engine version — redundant given the digest, but useful in incident reports.
3. Flags and config — env vars like VLLM_ATTENTION_BACKEND change effective behavior per engine version (MRV1 fallback!), so config alone pins nothing.
4. Weights hash — commit the checksum; a floating revision re-quantized differently is not the artifact you benchmarked.
5. Kernel/dependency pins — SGLang 0.5.19 requires FlashInfer 0.6.18 with no fallback on named paths ¹.
6. Eval baselines — golden outputs you diff against, so "same weights, new engine" is tested for behavior, not just throughput.

Six artifacts, not two — every release above touches a subset of them.

Pin the digest, not the tag. Tags are mutable: vllm/vllm-openai:v0.29.0 can be re-pushed; latest floats mid-deploy. Pin the immutable form:

image: vllm/vllm-openai@sha256:<digest>   # what you deploy

Retrieve it once (docker inspect --format '{{index .RepoDigests 0}}' or crane digest), store it in the manifest, and treat any digest change like a version bump: changelog, canary, rollback plan.

Upgrade or freeze? A decision framework

The right posture is asymmetric: fast for security and model support, slow for performance.

Upgrade immediately when a release fixes a security issue (anything touching request parsing, file loading, or LoRA paths on the endpoint) or adds a model you need — the cost of not upgrading exceeds the churn tax by construction.

Freeze deliberately when in production serving and the notes contain neither. Your engine choice was made with benchmarks true for the pinned version; nothing upstream improves your deployed reality without re-running them on your hardware — the vLLM vs SGLang guide makes the same point at selection time. A three-to-six-month pin beats weekly thrash on everything except security.

A/B a runtime with the same weights — the core skill this cadence demands, with MRV2 as the worked example:

1. Bring up the candidate on the same node class, same weights digest, same flags.
2. Replay recorded traffic or a fixed benchmark with pinned seeds — the release notes were validated on someone else's traces.
3. Diff latency distribution (p50/p99), throughput at real concurrency, and output equivalence — a scheduler or sampling change can alter generations without any error anywhere.
4. Check which runner actually ran: for vLLM 0.29.0, confirm in the logs whether MRV1 silently engaged ². If it did, your A/B measured the old runner — and says nothing about the path you run after v0.32.

Promote only on all passing, keep the old digest warm for instant rollback, and budget the 33-minute recovery into the window.

The critical view

Release-blog benchmarks are cherry-picked kernels, not your end-to-end. vLLM's notes quote kernel speedups (6.6–7.6× on a Mamba metadata kernel, 12.9–25.2% on a GEMM path); SGLang quotes up to 1.52× decode throughput for a specific AMD kernel on specific hardware ^{2 1}. Honest about their scope — but single-kernel measurements on vendor-adjacent hardware, while your bill is end-to-end tokens per second on your fleet. The only benchmark that binds is the one you run.

"Default to vLLM" is now a moving-target statement. "vLLM" in the sense of what your flags do changed on September 9, when MRV2 became default with per-feature fallback rules, and changes again by v0.32, when MRV1 is removed. Any guide — including this site's — is a statement about a version window.

Cadence-as-strategy, honestly read. The engine war benefits users in aggregate: features reach everyone in weeks, not quarters. But aggregate benefit and individual cost are different ledgers. Capturing the benefit requires absorbing the upgrades, and the 74-releases-per-year evaluation burden does not shrink because you wish it to. Cloud providers amortize absorption across thousands of tenants; a self-hoster with pinning discipline pays it deliberately, in windows they choose.

Three breaking releases in ten days is 2026's normal, not its crisis. The crisis is running this stack as if the 2024 cadence still applied: unpinned images, "engine version + weights" as if that pins anything, benchmarking as a one-time act. The stack that survives pins the digest, records its six artifacts, and treats every upgrade as an A/B with a rollback path. Choose your engine with the architecture comparison; if you arrived via llama.cpp/Ollama, the discipline here is the tax on leaving ollama serve. Run the self-hosting math before committing GPU dollars.