As models outgrow a single GPU, the connection between GPUs becomes the bottleneck. NVIDIA NVLink and the NVLink Switch exist to remove that bottleneck, turning a tray of accelerators into something that behaves like one enormous GPU.
Key Takeaways
- Interconnect, not FLOPS, often limits large-model performance.
- NVLink replaces PCIe for GPU-to-GPU traffic inside the node.
- The NVLink Switch extends that to non-blocking all-to-all across many GPUs.
- The result: many GPUs that program like one.
01Why PCIe is not enough
PCIe was designed as a general-purpose peripheral bus, not a high-frequency GPU-to-GPU link. For collective operations like all-reduce — where every GPU must share gradients with every other on each step — PCIe bandwidth and latency quickly cap training throughput. NVLink provides direct, high-bandwidth, low-latency paths between GPUs, several times faster than PCIe.
02The NVLink Switch: all-to-all at scale
A point-to-point link helps a pair of GPUs; scaling to eight or more requires a switch. The NVLink Switch creates a non-blocking, all-to-all fabric so any GPU can reach any other at full speed. This is what lets an HGX system act as a unified memory and compute domain, and what makes large-model training practical rather than theoretical.
- Bandwidth — multiple terabytes per second of aggregate GPU-to-GPU throughput.
- Uniformity — every GPU pair sees the same low latency, so collectives don't stall.
- Memory pooling — GPUs address each other's memory as one large space.
03What it means for your build
If your roadmap includes models that span multiple GPUs, NVLink-class interconnect is not optional — it is the difference between linear and sub-linear scaling. Semifly designs node and rack topologies so the fabric never becomes the ceiling on your compute investment.
← Back to Insights
