Skip to content
Friendly disclaimer: flozi00 TechHub is a solo side-project next to a full-time job — personal learning notes, no official statements. Verify critical steps yourself.
flozi00 Logoflozi00 TechHub
ENDE

NVIDIA NVLink: Generations, Domains and Verification

Official NVLink bandwidth figures, the difference between HGX and NVL72 domains, and reliable Linux topology checks.

2 min readflozi00
nvidianvlinkgpuinterconnecthpchardware
Skip to the full article
Watch · explore · understand

Trace a GPU transfer through its actual NVLink domain

A switch connects supported GPU links; topology inspection and communication measurements explain what an application can use.

AI-assisted learning visualization · schematic article explanation

Step 1 / 4Read the unit and scope

An NVLink generation specifies link capabilities, while the system design determines domain size. Rack aggregate bandwidth is a sum of capacities, not a rate available to one GPU pair.

SystemCurrent stepSelect a block to explain it
Generation and domain limits
Per-GPU bidirectional rate differs from rack aggregate.
NVLink / NVSwitch domain
Routes communication within the supported local system.
  1. Generation and domain limits NVLink / NVSwitch domain
    system design limits
What to remember

NVLink provides a communication path and capacity; actual topology and workload measurements establish its benefit.

This is a schematic switched domain, not the topology of every NVLink system. Preliminary product figures retain that status, and a peer or collective benchmark does not predict an entire model’s speed.

Terms and the complete walkthrough
NVSwitch
Switching hardware that connects GPU NVLink links in supported systems.
Bidirectional bandwidth
The stated total capacity across both transfer directions.
Collective
A communication operation involving a group of GPUs, such as all-reduce.
  1. Read the unit and scope

    An NVLink generation specifies link capabilities, while the system design determines domain size. Rack aggregate bandwidth is a sum of capacities, not a rate available to one GPU pair.

  2. Follow one transfer

    A sender passes tensor data through the supported local interconnect to a peer. The arrows show one transfer direction; published bidirectional rates include capacity in both directions.

  3. Inspect the installed topology

    Use topology and link-state queries to learn which paths actually exist and are active. A GPU family name alone does not prove every pair has the same route.

  4. Measure communication and the application

    Use peer-transfer or collective tests at relevant message sizes, then measure the model. The benefit depends on how much of its time is spent communicating and whether that traffic overlaps computation.

All connections

  • Generation and domain limits → NVLink / NVSwitch domain: system design limits
  • Sending GPU → NVLink / NVSwitch domain: send tensor data
  • NVLink / NVSwitch domain → Receiving GPU: deliver to peer
  • Inspect topology and link state → NVLink / NVSwitch domain: verify actual paths
  • NVLink / NVSwitch domain → Measure the communication: achieved transfers
  • Measure the communication → Measure the model workload: application relevance
Set your own pace. Pause at any time or select a step directly.

NVIDIA NVLink is a high-bandwidth GPU interconnect. NVSwitch connects multiple NVLink-equipped GPUs within a supported system design. The link can reduce communication time for some multi-GPU workloads, but neither the presence of NVLink nor a datasheet bandwidth guarantees an application speedup. GPU model, system topology, data size and software determine what is actually achieved.

Bandwidth and domain size

NVIDIA's NVLink product specifications give these maximum bidirectional bandwidths per GPU:

GenerationPlatformMaximum per GPU
NVLink 3Ampere A100600 GB/s
NVLink 4Hopper H100900 GB/s
NVLink 5Blackwell1,800 GB/s
NVLink 6Rubin3,600 GB/s, preliminary

The generation alone does not say how many GPUs are in one domain. For example, HGX B200 is an eight-GPU baseboard with fifth-generation NVLink. GB200 NVL72 connects 72 GPUs in a rack-scale NVLink domain and lists 130 TB/s aggregate NVLink bandwidth. That aggregate sums link capacities across the rack (approximately 72 × 1.8 TB/s); it is not a per-GPU rate or a measured collective throughput.

NVIDIA specifies 3.6 TB/s bidirectional per GPU and 260 TB/s aggregate for Vera Rubin NVL72. Its own specifications label the Rubin figures preliminary. Claims about larger future domains should be tied to a particular announced system and its status, not assigned to every NVLink generation.

Inspect a Linux system

The nvidia-smi documentation provides topology and NVLink queries:

bash
nvidia-smi topo -m
nvidia-smi topo -p2p n
nvidia-smi nvlink -s

topo -m shows the reported GPU/NIC topology and path labels. topo -p2p n checks NVLink peer-to-peer capability between GPU pairs. nvlink -s reports link state and, for active links, link bandwidth. The output depends on driver, GPU and system architecture; do not infer that all GPU pairs have the same path from a product family name alone.

For achieved GPU-to-GPU communication, use a workload-level measurement. NVIDIA's nccl-tests checks collective correctness and performance; the CUDA p2pBandwidthLatencyTest sample checks peer transfers. Record GPU count, placement, message size and the units used. A peer transfer test does not by itself predict end-to-end model training or inference speed.

Tensor parallelism and expert parallelism can exchange data frequently enough to benefit from a high-bandwidth local domain. A data-parallel job may communicate at different intervals, and a workload with little GPU-to-GPU traffic may see little benefit. Communication libraries may choose different algorithms for different message sizes and topologies.

Do not change GPU power limits, driver branches or kernel settings merely to “enable NVLink.” First confirm the hardware supports it, inspect topology and measure the actual workload. For the difference between rack-local and inter-rack bandwidth, see Scale-up vs Scale-out.