Skip to the board

Machine Learning

NVIDIA Opens Vera Rubin AI Factory Designs and Networking Standards at OCP

NVIDIA’s OCP contributions span rack designs, open operations software, power specifications and networking for large AI systems.

NVIDIA Opens Vera Rubin AI Factory Designs and Networking Standards at OCP

NVIDIA’s Open Compute Project (OCP) announcements address a practical scaling problem: building large AI systems requires coordinated choices across racks, power, cooling, networking and operations. At the OCP Global Summit in San Jose, the company contributed Vera Rubin rack and compute tray designs through its open Vera Rubin and MGX architectures, alongside software and specifications developed with industry partners.

The scope runs from facility design and utility-grid behavior to the connections between accelerators. That breadth reflects NVIDIA’s view that system-level coordination can improve deployment speed, power efficiency and tokens per watt. The announcements also set out a wider ecosystem’s direction: common designs and interfaces that manufacturers, operators and software teams can build on.

Rack designs target modular, higher-power deployments

NVIDIA joined industry contributors in submitting a third-generation MGX AI data center reference design to OCP’s Open DC Initiative. The updated MGX 1.2 specification extends the architecture with rack-scale blind-mate liquid cooling, interoperable rack interconnects and a liquid-cooled 5,000A busbar designed for higher-power systems. It also supports native 19-inch equipment and higher-density rack configurations.

🔗

MGX 1.2 adds configurable I/O bays, fanless liquid-cooled compute trays and standardized power and cooling interfaces. The aim is to give partners a shared modular base for accelerated computing while making assembly, testing and field service easier. The design supports rack deployments up to 250kW, according to NVIDIA.

These specifications matter because rack integration affects how quickly infrastructure can be manufactured, installed and serviced. Shared interfaces can help partners reuse components and simplify integration across deployments. The practical gains will depend on implementation: operators still need compatible equipment, facility capacity and service processes suited to liquid-cooled systems.

A separate Vera CPU rack design expands MGX beyond accelerated computing. NVIDIA positions the Vera Rubin NVL72 as a platform for large-scale training and inference, while the Vera CPU rack is intended for agent workloads involving tool calls, orchestration and sandboxed execution. It uses the third-generation MGX rack architecture, bringing compute, networking, power and cooling into a common modular design.

That separation points to an infrastructure split between accelerator-heavy model workloads and CPU resources for coordinating tasks and interacting with applications and services. The shared rack architecture could make those specialized systems easier to deploy within one broader environment.

Open specifications cover facilities and grid response

NVIDIA, AMD, Google and Meta published Open Data Center Specification v0.7, a blueprint for AI factory facility design covering power architecture, cooling and physical layout. A separate OCP Grid Resilience Specification, published by Google, Meta, Microsoft and NVIDIA, describes how large AI installations should respond to utility-grid disturbances.

The grid specification calls for systems to stay connected, respond automatically and return to near-full load in less than two seconds. That target makes grid behavior part of data center design, rather than an operational detail left until after construction. Open specifications give equipment makers, colocation providers and operators a shared design target; they may also reduce the need to work from separate customer-specific requirements.

NVIDIA says its DSX reference design provides an implementation compatible with both specifications, including facility layout, power architecture and software that monitors and responds to grid signals. Adoption will depend on how well the designs fit local facilities and the equipment operators choose.

DSX software addresses cluster operations

NVIDIA expanded its DSX AI factory platform with open source components for configuring networks, checking cluster readiness, managing racks and provisioning bare-metal infrastructure. DSX is intended to coordinate computing, networking, power, cooling, facilities and software; DSX OS supplies the operating layer for repeatable system-wide operations.

NVIDIA says the components draw on its experience operating DGX Cloud and are intended for cloud providers and software partners building multi-tenant platforms. The company describes cluster provisioning as taking hours instead of days, but the source provides no benchmark methodology for that comparison. Developers can find the components through the DSX AI Factory GitHub page.

Power design prepares for denser racks

Visionbay introduced K-1, an AI factory design built around 800V DC distribution and planned for future Vera Rubin deployments. The design combines NVIDIA’s 800 VDC reference architecture with power and thermal technologies from Vertiv.

As rack power rises, conventional low-voltage distribution can require more copper, cabling and conversion equipment. K-1 uses cluster-level power centers to centralize AC-to-DC conversion and carry 800V DC closer to compute. Visionbay describes a phased approach, allowing operators to introduce the architecture where it provides the most benefit instead of redesigning an entire facility at once.

The design pairs power distribution with liquid cooling and modular expansion. Its value will turn on deployment details: whether facilities can accommodate the new distribution model, how operators phase conversion and how the system performs with future hardware. The announcement outlines a design direction, not measured operating results.

Networking announcements focus on bandwidth and resilience

Meta plans to use NVIDIA Spectrum-6 102.4Tb/s Ethernet switches in Vera Rubin AI factories. NVIDIA says Spectrum-6 provides twice the switching capacity and per-lane bandwidth of Spectrum-4, with a lossless Ethernet fabric operating at 200G per lane. The design also aims to isolate traffic so one job’s network activity doesn’t slow another’s.

Meta’s N-IceCube switch, based on Spectrum-6, fits into IcePack, a 100T liquid-cooled rack, and runs Meta’s open-source Facebook Open Switching System (FBOSS). FBOSS uses the Switch Abstraction Interface, an OCP standard NVIDIA helped develop, to control switch hardware across vendors. The companies describe the architecture as suitable for AI systems with more than 100,000 GPUs, including those training and serving mixture-of-experts models.

Spectrum-X Multiplane, enabled by Spectrum-6 switches and ConnectX-9 SuperNICs, distributes GPU traffic across as many as eight independent network planes. NVIDIA says this can support larger systems with fewer switch tiers. Multiple paths can also provide alternate routes when a component fails, though traffic may continue at reduced bandwidth. Meta and NVIDIA say they tested improvements across on-premises and cloud environments. Meta’s networking director Omar Baldonado highlighted the ability to extend FBOSS to Spectrum hardware and use the same software stack across fabrics.

NVIDIA and Cisco also introduced USFP, a pluggable optical module form factor designed for 400Gb/s-per-lane networking. With approximately the footprint of the OSFP form factor used for 200Gb/s-per-lane optics, USFP is designed to carry 16 full-duplex lanes, with plans to scale to 32. NVIDIA and Cisco say that would deliver four times the bandwidth density of traditional 200Gb/s-per-lane pluggable optics and roughly 1.3 times greater power efficiency. The companies are working with TE Connectivity and Amphenol on cages, connectors and modules, with plans to advance the design through a multi-source effort.

Finally, NVIDIA and Google announced that future ConnectX-10 SuperNICs and BlueField DPUs will support Google’s Falcon Transport. Falcon is designed to transfer data reliably over Ethernet, using multipath load balancing, programmable congestion control, hardware recovery for dropped packets and per-connection security. NVIDIA says its hardware acceleration and DOCA software will let NVIDIA and Google network devices communicate using a shared transport while applications retain standard RDMA interfaces.

Taken together, the OCP work links open rack designs and operating software with facility, power and network specifications. The next test is whether partners can implement these interfaces consistently across real deployments—and show that the promised gains in deployment speed, efficiency and resilience hold under production workloads. More details are available from NVIDIA at OCP.

Source: NVIDIA Writers · blogs.nvidia.com

Hanging nearby