Self-hosting DeepSeek V4 Pro: the actual math

DeepSeek-V4-Pro shipped with the two most seductive numbers in open-weights history: 1.6 trillion parameters and 1M-token context, under an MIT license. The internet's takeaway — "a frontier model you can run at home" — is wrong by roughly three orders of magnitude. This guide is the arithmetic that separates the two claims: total parameters set your memory bill, active parameters set your speed, and the 1M context sets your KV-cache bill. This is a multi-accelerator deployment; the required configuration depends on usable memory, topology and serving software.

Every figure below is either quoted from DeepSeek's own primary sources (release note, model card, tech report, config.json) or computed in the open, with assumptions labeled. Hardware reference figures (H200 = 141 GB @ 4.8 TB/s, H100 = 80 GB @ 3.35 TB/s, B200 = 180 GB @ 7.7 TB/s, MI355X = 288 GB @ 8.0 TB/s) are the site-audited constants from the VRAM deep dive.

The official config, pinned

From the model card and release note — not from anyone's benchmark ^{1 2}:

The instruct checkpoint you actually download is the FP4+FP8 mixed one: its 64 safetensors shards total 865 GB as published on Hugging Face ⁴ — before a single byte of KV cache, activation buffers, or CUDA graphs.

The MoE asymmetry: weights are total, compute is active

Mixture-of-Experts means each token is routed through a small subset of the network. DeepSeek's config: 384 routed experts, 6 selected per token, 1 shared expert. So per token, only roughly 49B of the 1.6T parameters are touched. This creates the asymmetry that makes V4-Pro look deceptively cheap:

Weights VRAM scales with total parameters (1.6T) — every expert must sit in memory even if most are idle, because routing is data-dependent and unpredictable. Serving FLOPs and per-token weight reads scale with active parameters (49B) — the same byte level as a mid-size dense model. This is the entire "1.6T sounds like 49B" pitch: the marketing number (49B active, ~30 GB of bytes moved per token) and the memory number (1.6T, ~0.9–3.2 TB resident) are both true, simultaneously, and only one of them fits in your budget. It is the same quantization-vs-bits-per-weight math as for dense models — the twist is which parameter count each resource follows.

The asymmetry is real, not a trick: it is why DeepSeek can serve this at API prices we will pin below. But it transfers compute cost to you and keeps memory cost in full. Your 8-GPU node sits at ~77–88% of its memory just holding weights to win back a per-token speed of a small model.

Weight bands: what fits on what

Weight bytes at each precision (using $P_{\text{total}}$, per the formula above), plus the ~15% serving overhead (activation buffers, communication scratch, CUDA-graph allocations — the same overhead band our inference calculator uses) before any KV cache:

Read that right-hand column carefully: not one shipping GPU holds even the FP4 weights — the 288 GB MI355X holds less than a third of the released checkpoint. The node math only closes because 8-GPU platforms pool memory via tensor/expert parallelism, and every one of those inter-GPU hops costs bandwidth — on H200 NVLink (~900 GB/s bidirectional per GPU) experts cross GPUs constantly, which is why MoE serving is far more parallelism-sensitive than dense models.

And for a 1M-context model, fp16 was never in the running: the released model is FP4+FP8 natively, quantizing it back up to fp16 buys quality margin nobody has measured yet and triples a bill you already cannot pay. The realistic bands are the last two rows: the published checkpoint on an 8×H200-class node (~994 GB total, KV budget ~50–130 GB) or fp8 on 8×MI355X / larger clusters.

The KV chain: where 1M context explodes

Weights are the fixed cost; KV cache is the per-user cost, and it scales linearly with context. The general formula is the same one derived in our KV-cache guide:

V4-Pro does not publish bare per-token KV bytes the way a GQA model does — its CSA/HCA hybrid compresses the cache along the sequence dimension with per-layer ratios (128×, 4×, plus 0 on some layers, all present in config.json ⁵). What DeepSeek does publish, as their own headline efficiency claim: at 1M context, V4-Pro needs only 10% of the KV cache of DeepSeek-V3.2, while using only 27% of the single-token FLOPs ³. V3.2 uses V3's MLA scheme at 68.6 KiB/token in bf16 (the same figure our KV guide derived from V3's config ⁶). So the honest, labeled estimate:

— an estimate derived from their published ratio applied to a known V3 config, not a figure they print directly. Now the table for one user:

Two readings, and you need both:

• The compression is genuinely remarkable. A 1M-token conversation for 7.4 GB would have been ~74 GB on V3.2 and would be measured in terabytes on an uncompressed GQA model at this scale. The architecture is doing real work.
• 1M context is still a full H200 of memory per couple of users. On the only realistic node (8×H200, ~130 GB free after the FP4 checkpoint + overhead), a fully-utilized 1M-context user costs 7.4 GB — but 32 concurrent 128k users cost $32 \times 0.92 = 29.4$ GB, and the Think-Max mode DeepSeek itself recommends at ≥384K context ² triples that per user. Your concurrency ceiling on the flagship config is dozens, not hundreds, unless you buy more nodes.

The prefill bill matters too: processing 1M input tokens means ~1M × (49B active × 2 FLOP/param) ≈ $10^{17}$ FLOPs of compute — a FLOP estimate, not a time prediction. Attention, routing and achieved utilization must be included to estimate prefill latency; large prompts also allocate KV state and compete with other requests for memory.

Throughput ceiling: bandwidth ÷ bytes-moved

Decode is memory-bound (the KV guide's arithmetic model): each generated token requires reading the active weights from HBM. With $P_{\text{active}} = 49$B:

Aggregate ceiling includes KV-read overhead: at full 1M context (7.37 GB KV re-read per token, effective) it drops to ~681 tok/s on 8×H200 fp8.

Caveats, stated: these are ceilings, not benchmarks — they ignore attention FLOPs, routing/communication overhead (large for 384 experts spread over 8 GPUs), kernel launch and dequantization cost. Actual throughput must be measured for the model, batch size, runtime and interconnect; no fixed efficiency percentage follows from this roofline. And per-user latency is what users feel: batching N users on one weights-read amortizes bandwidth but divides per-user speed by N. The FP4 column's 26.5 GB is our estimate (49/1600 of the 865 GB checkpoint). The important fact survives all caveats: the node behaves, per token, like a 30–50 GB model that takes 900+ GB to store.

The API you are competing against

DeepSeek's published peak rates for deepseek-v4-pro, checked 29 September 2026, are $1.32 per million cache-miss input tokens, $3.96 per million output tokens, and $0.044 per million cache-hit input tokens. Off-peak rates are half the peak rates; prices can change.⁷ For an illustrative 3:1 cache-miss input:output mix, one million total billed tokens costs:

Compare the same unit on both sides. Assume a rented 8×H200 node costs $22/hour and delivers 300 output tokens/second sustained, with exactly three input tokens per output. That corresponds to 1,200 total tokens/second, or 4.32 million total tokens/hour. Ignoring the extra compute and scheduling needed for prefill, its optimistic cost is $22 / 4.32 ≈ $5.09 per million total tokens. Against the $1.98 peak API example, that is about 2.6×, not 10×. The earlier comparison divided by output-token throughput and compared the result with a blended input-plus-output API price; those units were inconsistent.

At the theoretical 784 output-token/second H200 bandwidth ceiling above, the same mix would yield 3,136 total tokens/second and an optimistic $1.95 per million total tokens. The exact cost-parity point under these assumptions is about 772 output tokens/second (22 × 10^6 / (1.98 × 3600 × 4)). That is very close to the roofline, which omits communication, attention, prefill, power and utilisation losses. A measured deployment would normally need a lower hourly cost, cache-hit-heavy traffic, off-peak comparison, or a different workload mix to change this result. No universal self-hosting break-even follows from these figures.

These calculations are illustrative. The actual decision requires a hardware quote, measured throughput including prefill, the real input/output and cache-hit mix, utilisation, support and operational costs, and a current API price. Data-location or control needs may also matter independently of token price.

What "open weights" does and does not buy you

The critical view, primary-source edition:

• Weights are open; the recipe is not fully reproducible. MIT-licensed weights, yes. But the 32T+ token pretraining corpus is unpublished, the full post-training pipeline (domain experts, GRPO reward models, on-policy distillation teachers) is described in the tech report ³ only at the level you can read above. "Open" here means usable and modifiable, not reproducible — you cannot retrain or even fully audit what you are deploying.
• DeepSeek never claimed "beats every frontier model." Their own model card comparison tables ² show V4-Pro-Max losing to Opus-4.6 Max and Gemini-3.1-Pro on most knowledge and agentic benchmarks (e.g. MMLU-Pro 87.5 vs Gemini's 91.0; Terminal Bench 67.9 vs GPT-5.4's 75.1), and winning on some coding and math ones. Claims of "beats frontier" circulating in third-party posts are community leaderboards and vibes — the release note itself says "performance rivaling the world's top closed-source models" and, notably, "please rely only on our official accounts for DeepSeek news" ¹. DeepSeek's own honesty here is unusually good; borrow it.
• Pin the version actually used. The original V4 announcement and model card describe a preview, but DeepSeek announced V4-Pro general availability in August 2026 and its pricing page lists DeepSeek-V4-Pro-0813 as the API version.⁸⁷ An open-weight checkpoint and an evolving API endpoint should not be assumed byte-identical.
• The efficiency claims are DeepSeek's, not yet independently verified at the byte level. The 10%-of-V3.2 KV and 27%-of-FLOPs figures come from their tech report's own Figure 1 estimates. Our KV table above inherits that uncertainty and says so.

The takeaway is not that V4-Pro is overhyped — on the primary sources, it is an extraordinary piece of engineering. The takeaway is that open weights transfers you a memory bill and an ops problem, not a frontier model you can park next to your desk. The 49B is what you pay per token; the 1.6T is what you buy upfront; and 1M context is the line item that decides your fleet size. Run your own numbers in the inference calculator before quoting anyone's cluster.