Why NVSwitch Could Become InfiniBand’s Scale-Up Equivalent in AI Networks

From InfiniBand to Ethernet

In the early development of PC and server interconnects, two competing I/O technologies eventually reached a compromise during the dot-com boom. That agreement helped establish InfiniBand ports and switched fabrics as a potential I/O bus for computing.

The dot-com bust changed the industry’s priorities. Ethernet became the preferred technology for point-to-point connections, including links between PCs, servers, and networks, as well as for what is now called scale-out networking across multiple machines. With budgets under pressure, the industry also adopted updated PCI-X and later PCI Express buses for peripherals.

InfiniBand eventually returned as a high-performance, low-latency interconnect through the work of Voltaire and Mellanox Technologies. Mellanox supplied switch ASICs and network interface cards, while Voltaire built switches for high-performance computing and ModSim workloads. Mellanox acquired Voltaire for $218 million in November 2010, consolidating its position in InfiniBand switching.

InfiniCon Systems, founded by former Unisys engineers, also entered the market. After several corporate changes, it became QLogic, Mellanox’s only significant InfiniBand competitor. Intel acquired QLogic’s InfiniBand business for $125 million in January 2012. That team was later spun out of Intel and became Cornelis Networks in September 2020.

Other companies also participated in the market. Cisco Systems acquired InfiniBand switch vendor TopSpin in 2005. Sun Microsystems, guided by co-founder Andy Bechtolsheim, used InfiniBand for clustering in its “Constellation” HPC systems and developed its own InfiniBand ASICs and NICs. Oracle continued that approach after acquiring Sun Microsystems for $5.6 billion in April 2009. Oracle also acquired InfiniBand switch maker Xsigo Systems before ultimately choosing Ethernet as its platform for scale-up, scale-out, and scale-across networking.

Ethernet Moves Into AI Networking

The Ultra Ethernet Consortium, founded in July 2023, aims to combine InfiniBand-like bandwidth, latency, quality-of-service features, and adaptive routing with Ethernet’s scalability, multitenancy, and broad compatibility. Its goal is to create scale-out networks capable of supporting AI clusters with up to 1 million endpoints.

Ethernet is also moving into the scale-up domain. Nvidia’s NVSwitch interconnects connect GPU memories into a coherent shared-memory space for 72 accelerators, with an expansion path to 576 devices when additional latency hops are acceptable.

InfiniBand remains a low-latency leader and retains an advantage in some workloads. However, large hyperscalers, cloud builders, and AI model developers are increasingly considering Ultra Ethernet technologies. Ethernet ASIC vendors including Broadcom, Cisco Systems, and Nvidia are also working to reduce switch-hop and end-to-end latency.

InfiniBand is likely to remain available for a long time, particularly through Nvidia and specialized HPC systems. However, Ethernet’s scale advantages and compatibility with campus, edge, and datacenter networks could confine InfiniBand to a smaller market. Nvidia’s NVSwitch products for scale-up networking and Spectrum-X Ethernet products for scale-out networking are positioned to become major parts of its networking business. Demand from customers, rather than only vendor strategy, is driving this shift. Enterprises and AI model developers generally favor Ethernet, including Meta Platforms, which previously used InfiniBand in earlier AI clusters.

Ethernet-Based Alternatives to NVSwitch

A long-standing pattern in networking is that Ethernet adopts useful ideas from other fabrics and applies them across both enterprise and specialized HPC or AI environments. The same pattern is emerging in scale-up networks that connect GPUs and other accelerators.

Very fast Ethernet is already being used to transport other memory-sharing approaches. One example is UALink, founded in May 2024 by AMD, Broadcom, Cisco Systems, Google, Hewlett Packard Enterprise, Intel, Meta Platforms, and Microsoft. AMD is using Broadcom Tomahawk 6 Ultra Ethernet switches to run the UALink protocol inside its “Helios” racks.

UALink is a memory-atomic protocol for connecting GPUs and other XPUs. The protocol can also support shared memory between CPUs and DPUs if system designers choose to extend it in that direction.

Another effort is the ESUN protocol, supported by Meta Platforms and Microsoft, along with AMD, Arista Networks, Arm, Cisco, Hewlett Packard Enterprise, Marvell, Nvidia, OpenAI, and Oracle. Broadcom participates in the ESUN/SUE-T effort. The two names refer to different layers in an Ethernet scale-up stack, with SUE-T handling load balancing and other higher-level functions.

Broadcom has withdrawn from UALink and, based on current information, does not appear to be pursuing the NVLink and NVSwitch combination. That position could change if companies using Broadcom’s CPU and XPU design services decide to adopt NVSwitch as their rack-scale fabric and use Nvidia-style racks for GPUs and other accelerators.

Nvidia’s NVLink Fusion Strategy

Nvidia recently bought $3.5 billion in convertible bonds issued by Taiwanese system-on-chip maker MediaTek. As part of the agreement, MediaTek will gain access to Nvidia’s NVLink Fusion stack. This technology allows chip designers to add NVLink ports so their processors can participate in an NVSwitch shared-memory fabric.

Nvidia does not currently sell NVSwitch ASICs and systems as independent products. Customers must include a Grace or Vera CPU, or an Nvidia Blackwell or Rubin GPU, in the configuration. Nvidia also expects customers to purchase racks containing NVSwitch as the scale-up network.

Nvidia could eventually choose to sell NVSwitch networking to customers whose CPUs, GPUs, and other accelerators include NVLink ports. Such a model would require additional software support. For now, Nvidia’s public position is that this broader access is not permitted.

The Long-Term Scale-Up Outlook

Nvidia currently accounts for an estimated 95 percent of GPU revenue and approximately 75 percent of combined GPU and XPU revenue. As a result, it holds a dominant position in scale-up networking through NVLink and NVSwitch. That share is expected to decline only modestly through calendar 2027.

Hyperscalers, cloud builders, AI model developers, and large enterprises generally prefer widely adopted standards that allow hardware from multiple suppliers to work together. NVLink Fusion could delay the wider adoption of UALink and ESUN/SUE-T for another generation or two, but industry demand for interoperability remains strong.

Major technology companies have the resources to create standards for their own hardware and may eventually converge on a common approach. A similar process occurred with 25 Gb/sec signaling for 100 Gb/sec and faster Ethernet in 2014, when Google, Microsoft, and Arista Networks moved ahead after becoming dissatisfied with the IEEE’s slower approach based on 10 Gb/sec signaling and ten lanes.

UALink could compete directly with NVLink and NVSwitch and, in theory, scale almost twice as far. The UALink 1.0 specification, launched in April 2025, supports up to 1,024 XPUs or GPUs through a single level of UALink switching. NVSwitch is believed to require two switching tiers to connect 576 GPUs in a shared-memory system.

The UALink and ESUN/SUE-T groups could eventually combine their efforts into a common standard, potentially called UALink 2.0. The development of CXL provides a precedent for competing groups reaching an agreement on a memory-coherency standard.

Latency will be an important factor. InfiniBand switch hops typically range from 100 nanoseconds to 120 nanoseconds, while the UALink specification expects a port-to-port hop of approximately 100 nanoseconds. UALink does not mandate a specific latency, leaving vendors to optimize their implementations.

NVSwitch hop latency has not been publicly confirmed. It is rumored to be closer to Ethernet than to InfiniBand. If that proves accurate, UALink could gain an advantage by delivering InfiniBand-like latency, particularly if it achieves those results using Ethernet switches.

If UALink becomes the de facto scale-up standard, Nvidia could respond by building competitive UALink switches and ports. In that scenario, Nvidia could still generate substantial revenue from its existing NVLink and NVSwitch business, even if the industry ultimately adopts an Ethernet-based standard for scale-up networking. The same pattern could apply to InfiniBand, which may remain an important Nvidia product line even as Ethernet becomes the preferred technology for large-scale networks.