Turn useful capacity and real costs into a comparison
Separate modeled token capacity from ownership, rental and API billing, then compare task quality before choosing.
AI-assisted learning visualization · schematic article explanation
A roofline estimates potential decode capacity, while utilization describes how much is used. Replace ideal token rates with measured workload throughput before treating the period’s token volume as a purchase forecast.
The calculator below compares a simplified decode capacity model with editable ownership, rental and API-price assumptions. It is a scenario tool: a cost result is meaningful only after entering a complete hardware quote, a real rental offer and measured serving throughput.
Physics and capacity
For small-batch dense decode, memory bandwidth divided by weight bytes is a weight-read throughput ceiling if weights and KV cache fit. The compute approximation is effective tensor FLOPS divided by twice the active parameter count. Both omit attention, prefill, routing overlap, scheduling, communication and runtime allocation details. The displayed batch crossover B* is where these two modeled ceilings meet; it is not an observed scheduling optimum. See the inference-math guide and break-even model.
Cost assumptions
GPU acquisition and cloud hourly prices in the tool are illustrative inputs, not verified market quotes. Replace them with a full-system purchase quote and a provider rate for the same GPU form factor, contract and region. The power calculation uses a simple assumed idle-to-peak curve and only GPU/board watts; the displayed 36-month depreciation excludes financing, non-GPU power and cooling. The output-token capacity estimate assumes continuous decode at the roofline with a user-selected efficiency and utilization. Prefill and traffic bursts can require additional capacity.
The API menu uses uncached standard text-token rates checked on 29 September 2026 against the vendors' own documentation: OpenAI, Anthropic, Google and xAI. Cached input, longer contexts, region, batch processing and tools can change the bill. The same selected API is used as a price reference for every open-weight row; the table makes no model-quality equivalence claim. Evaluate candidate models on your tasks before comparing cost per useful answer.
Economics
LLM inference economics beta
From physics to euros: how memory and compute cap your throughput, which model fits which hardware, what the hardware costs, and when owning beats renting — including the comparison against proprietary API pricing.
Token yield from this hardware
Operating point: min(B*, VRAM-capable batch), FP8 KV, continuous serving.
Monthly cost of owning
Scenario: acquisition amortized over 36 months; financing cost and non-GPU system power are excluded.
Which models fit which hardware (FP8 (1 B), 1× GPU)
| Model | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-4B-Instruct-25074B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Qwen3-14B14B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Qwen3-32B32.76B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ |
| Qwen3-VL-8B-Instruct8B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Qwen3-30B-A3B30.5B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | ✓ | ✗ | ✓ |
| Qwen3-Next-80B-A3B80B | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ | ✓ | ~ | ~ | ✗ | ✗ | ✗ | ✗ | ~ |
| Qwen3.8-Flash-Next180B | ✓ | ✗ | ~ | ✗ | ✗ | ✓ | ✓ | ~ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Qwen3.8-27B27B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ~ | ✓ | ✗ | ✓ |
| Qwen3-235B-A22B235B | ~ | ✗ | ✗ | ✗ | ✗ | ✓ | ~ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Llama 3.3-70B70B | ✓ | ✓ | ✓ | ✓ | ~ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ~ | ✗ | ✓ |
| gpt-oss-120B-A5B117B | ✓ | ✓ | ✓ | ~ | ✗ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| DeepSeek V3.2-671B-A37B671B | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| DeepSeek V4-Flash-0731284B | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| DeepSeek V4.1-Flash748B | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| GLM-5.2753B | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ |
| Qwen3.6-35B-A3B35B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ |
| Nemotron Nano 9B v28.9B | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
✓ weights leave ≥15% raw capacity · ~ less than 15% · ✗ weights exceed capacity. KV and runtime memory are not included. Click a GPU header to select it. Format factor (3%) included.
Renting the same GPU in the cloud
Hourly rental is an editable scenario input, not a provider quote. Billing minimums and contract terms vary.
API vs own hardware for your workload
Uncached standard API rates checked Sept 29, 2026; edit for your plan. Hardware count is a decode-only lower bound, not a quality comparison.
Decode-only hardware cost vs selected API, per 1M output tokens
Open-weight cost = own-hardware cost at 70% utilization, FP8 (1 B) weights, FP8 KV.The selected API is a price reference for every row; model quality, prompt processing and operations are not matched.
| Open-weight model | Selected API reference | API in / out ($/Mt) | GPU size needed | Own $/Mt | API $/M output tokens at 30% out) | Verdict |
|---|---|---|---|---|---|---|
| Qwen3-32B (dense) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.03 | $5.5 | own −99.44% |
| Qwen3-VL-8B-Instruct (dense + vision) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.01 | $5.5 | own −99.89% |
| Qwen3-14B (dense) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.01 | $5.5 | own −99.81% |
| Qwen3-4B-Instruct-2507 (dense) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0 | $5.5 | own −99.95% |
| GLM-5.2 (753B-A40B, MLA + DSA) | Gemini 3.8 Flash | 0.75 / 3.75 | No fit ≤8 GPUs at 2k | — | $5.5 | — |
| DeepSeek V4.1-Flash (552B-A16B + 196B Engram) | Gemini 3.8 Flash | 0.75 / 3.75 | No fit ≤8 GPUs at 2k | — | $5.5 | — |
| DeepSeek V4-Flash-0731 (284B-A13B) | Gemini 3.8 Flash | 0.75 / 3.75 | 4× RTX PRO 60 | $0.26 | $5.5 | own −95.32% |
| Qwen3.8-Flash-Next (125B-A6B + PLE + MTP) | Gemini 3.8 Flash | 0.75 / 3.75 | 2× RTX PRO 60 | $0.31 | $5.5 | own −94.41% |
| Qwen3.8-27B (dense hybrid) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.02 | $5.5 | own −99.64% |
| Qwen3.6-35B-A3B (hybrid MoE + vision) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0 | $5.5 | own −99.95% |
| Qwen3-235B-A22B (flagship MoE) | Gemini 3.8 Flash | 0.75 / 3.75 | 3× RTX PRO 60 | $0.24 | $5.5 | own −95.61% |
| Llama 3.3-70B (dense) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.22 | $5.5 | own −95.91% |
| gpt-oss-120B-A5B (MoE, MXFP4) | Gemini 3.8 Flash | 0.75 / 3.75 | 2× RTX PRO 60 | $0.03 | $5.5 | own −99.5% |
| DeepSeek V3.2-671B-A37B (MoE + MLA) | Gemini 3.8 Flash | 0.75 / 3.75 | 8× RTX PRO 60 | $0.16 | $5.5 | own −97.05% |
| Qwen3-Next-80B-A3B (hybrid MoE) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.04 | $5.5 | own −99.35% |
| Nemotron Nano 9B v2 (hybrid Mamba) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.01 | $5.5 | own −99.88% |
| Qwen3-30B-A3B (MoE) | Gemini 3.8 Flash | 0.75 / 3.75 | 1× RTX PRO 60 | $0.01 | $5.5 | own −99.81% |
GPU purchase and rental prices are illustrative inputs, not verified offers. Enter your complete-system quote and rental rate. API prices are uncached standard rates checked Sept 29, 2026; long-context, regional, caching, batch and tool charges are excluded. Hardware throughput is a simplified decode roofline; prefill, attention, scheduling, networking, CPU work and non-GPU power are not modeled.
Related
- LLM Inference Calculator & Break-Even Analyzer — the batch-size curve behind the cost lines.
- LLM VRAM Requirements: A Mathematical Deep Dive — what decides whether your model fits the card you are pricing.
Limitations
- The efficiency and utilization inputs are user assumptions; there is no universal percentage of peak that every stack achieves.
- Purchase and rental defaults are scenario inputs. API rates need checking again before a buying decision.
- Prefill, scheduling, networking, CPU work, full-system power, taxes and financing are omitted; compare actual quotes and measured workloads.