Qwen's Qwen3-Next-80B-A3B-Instruct model card describes an Apache-2.0-licensed language model with 80 billion total and approximately 3 billion activated parameters per token. The two numbers describe different costs: active parameters are relevant to computation, while all stored weights still require memory unless parts of the model are offloaded or distributed.
Hybrid sequence layers and sparse experts
The model card specifies 48 layers arranged as twelve repetitions of three Gated DeltaNet layers followed by one Gated Attention layer. Each layer includes a Mixture-of-Experts (MoE) feed-forward component. There are 512 routed experts, with 10 selected per token, plus one shared expert. The router selects different expert weights for different tokens. It does not load only 3 billion parameters into GPU memory for the entire model.
The hybrid attention design targets long-context efficiency. The model card gives a native context length of 262,144 tokens and describes extension beyond that with additional configuration. These are supported context limits, not a promise that every deployment can run at that length: the available GPU memory, concurrency and serving implementation still matter.
What “A3B” does and does not predict
Sparse activation reduces the arithmetic required by the expert layers compared with evaluating every expert. It does not mean the whole 80B model has the latency, memory footprint or operating cost of a dense 3B model. Expert routing, access to stored weights, attention, long-context state, communication between GPUs and batch size all affect serving performance. The Qwen model card reports throughput and task evaluations for specific comparisons; they are not universal speed guarantees.
For a rough weights-only storage estimate, 80 billion parameters at two bytes each require about 160 GB in decimal units before runtime overhead. An eight-bit copy would require about 80 GB before scales, metadata, KV cache and buffers. Real memory use depends on the checkpoint's format and serving implementation. Quantizing to FP8 is an optional deployment choice that requires checking model quality and hardware support. The official model card does not describe FP8 as a defining property of the released checkpoint or claim “no significant accuracy loss” for every FP8 conversion.
Deployment
Qwen's model card documents serving with SGLang and vLLM, including separate options for multi-token prediction. It notes that MTP support and throughput depend on the implementation. The earlier version of this article incorrectly presented NVIDIA Transformer Engine as a model-card-supported serving option; the cited card does not make that claim.
Benchmark a chosen checkpoint, precision, context length and concurrency on the actual target hardware. The “80B-A3B” label alone is insufficient for a latency, GPU-memory or cost estimate.