Microsoft’s Clea Maia 300 XPU Emphasizes Bandwidth for AI Inference

Microsoft’s Maia 300 targets large-scale inference

AI inference remains expensive even with advanced GPU and XPU accelerators. Microsoft faces particularly high demand because it runs inference for OpenAI ChatGPT models, supports OpenAI model training, and is adding AI routines throughout its worker productivity and back-office applications.

Microsoft is therefore expanding its custom Maia accelerator family. As the Open Compute Project Summit 2026 begins, the company has released specifications for its future “Clea” Maia 300 inference accelerator.

Maia 300 introduces architectural changes intended to improve GenAI inference performance and scalability. The design targets larger agentic workloads that may involve dozens or hundreds of models. These models can operate through a mixture-of-experts foundation model, as separate independent models, or through a combination of both. In each case, efficient operation depends on sharing network and memory resources with low latency.

From Maia 100 to Maia 300

Microsoft introduced the first-generation “Athena” Maia 100 XPU in November 2023, alongside its “Cobalt 100” custom Arm server CPU. At that time, Microsoft disclosed limited architectural information about either chip.

The second-generation “Braga” Maia 200 was unveiled in January 2026 and has been entering production on Azure during the year. Scott Guthrie, Microsoft’s executive vice president responsible for Cloud and AI, described the multigenerational Azure Maia program as a foundational part of Microsoft’s AI infrastructure strategy.

Maia 200 chips are being used for Copilot inference and by the Microsoft AI team to develop new models. GitHub Copilot and Microsoft Foundry are expected to shift to Maia 200 by the end of 2026. GitHub Copilot had previously been supported by Maia 100. Maia 200 systems will run OpenAI models, Microsoft models, and open source models through Microsoft Foundry. The distribution of OpenAI’s GPT API workload between Maia 100 and Maia 200 has not been disclosed.

Microsoft has also said that Maia will be deployed in multigigawatt Azure installations. A gigawatt of Maia 300 systems would represent approximately 575,000 devices, based on 1.74 kilowatts for each accelerator and its share of the host, storage, and scale-up networking infrastructure. Microsoft says a Maia 300 rack consumes approximately 167 kilowatts and contains 96 XPUs.

Maia 300 package and memory architecture

The Maia 200 increased HBM capacity and bandwidth while reducing on-chip SRAM cache. It also introduced a substantially improved AI Transport Layer (ATL), Microsoft’s custom RoCE Ethernet implementation. Compared with Maia 100, Maia 200 provided 2.3 times more bandwidth, expanded the Maia cluster coherence domain by 2.7 times to 6,144 devices, and increased power consumption by 1.5 times.

By moving from 6-bit MX6 data formats to 4-bit MXFP4, increasing the number of compute cores by 1.5 times, and raising clock speeds, Maia 200 delivered 3.4 times the floating-point capability of Maia 100.

Maia 300 is not a conventional single-chip system. It is a package containing seven chips and eight HBM stacks mounted on an interposer. Microsoft refers to it as a system-on-chip in its technical material, but the package is more accurately described as a chip complex, or a system in a socket.

The design uses TSMC’s 2 nanometer N2P process for the compute elements and adopts a chiplet architecture. I/O and HBM controllers are separated from the compute and SRAM cache components. Like Maia 200, Maia 300 has four tile clusters, but each cluster contains twelve tiles.

The architecture also includes a reshaper and a tile synchronizer. These elements were introduced with Maia 200 but were not identified in earlier public information. They support data movement across the complete system, from the network to the server node and down to individual compute tiles.

The SRAM capacity of each tile and cluster has not been disclosed. Estimates in the available analysis place approximately 400 MB of SRAM across the central compute area and another 200 MB in the I/O chiplets for network and compute-data caching. These figures are estimates rather than Microsoft specifications.

The Maia 300 complex is estimated to provide approximately 166.2 TB/sec of aggregate SRAM bandwidth. That complements 18 TB/sec of HBM bandwidth across eight HBM4 stacks. The package contains 288 GB of HBM4, reportedly supplied by SK Hynix as with Maia 200.

Based on package images, known HBM capacities, and the process nodes used, the Maia 300 package is estimated to cover approximately 1,980 mm². That would make it about 2.4 times larger than the estimated 810 mm² Maia 200 package. The transistor count is estimated at approximately 280 billion, excluding HBM4 memory, with an uncertainty of roughly 10 percent.

Compute tiles and chiplets

Maia 100, Maia 200, and Maia 300 compute cores include a tile tensor unit (TTU), a tile vector processor (TVP), a tile data movement engine (TDMA), a tile control processor (TCP), and shared L1 SRAM.

Maia 100 placed four cores on each tile and sixteen tiles in a cluster, for 64 cores in total. Earlier information did not disclose the organization of Maia 200, but an arXiv paper published in August provided additional details.

Maia 200 uses tensor units with matrices twice the size of those in Maia 100. It therefore requires fewer cores to reach its target throughput than earlier estimates suggested. The chip has ten tiles per on-chip cluster and four clusters, for 40 cores total. The cores run at 2 GHz. Nine cores per cluster are active to improve yield on TSMC’s 3 nanometer N3P process.

Maia 100 and Maia 200 use monolithic designs that approach the reticle limit of lithography equipment. Maia 300 instead separates compute, SRAM, I/O, and HBM control functions across chiplets.

Networking and scale-up

The Maia 300 system uses a hierarchy of compute, memory, and networking resources. These resources are combined into pods and superpods to support large GenAI workloads.

Maia 100 used twelve 400 Gb/sec links. Nine links connected four Maia 100 accelerators tightly within a node, while three connected the node to a three-plane Ethernet fabric. The first network tier could connect 32 accelerators. A second tier expanded the system to 576 nodes and 2,304 Athena XPUs in one coherence domain.

Maia 200 implemented 28 Ethernet ports running at 400 Gb/sec, providing 2.8 TB/sec of aggregate bidirectional bandwidth. Four switch planes in the ATL 1.0 network used four ports per accelerator. The remaining ports connected four Maia 200 devices into a shared-memory system, with seven links per accelerator used for accelerator-to-accelerator connections, except for one link per device assigned to the first switch plane.

A Maia 200 rack contains twelve trays, each holding four accelerators, for 48 devices. Up to 128 racks can be connected through 128 first-tier switches and 32 second-tier switches, creating a coherence domain of 6,144 XPUs. The switches use 51.2 Tb/sec ASICs and provide 128 ports operating at 400 Gb/sec.

Maia 200 was designed to share HBM memory across its compute tiles. Its system board also tightly coupled four XPUs so that their memory could be accessed in a NUMA-style arrangement.

Maia 300 changes this organization. Each pair of compute tiles is associated with a pair of HBM4 memory banks in a NUMA-like arrangement. Memory is not explicitly shared across every tile and HBM4 stack in the socket. Workloads that remain within a particular memory region can therefore avoid some of the cost of broader memory access.

Only two accelerators are tightly linked to each other in the Maia 300 design, rather than four as in Maia 200. The Clea package contains four distinct HBM4 memory regions, connected through the ATL 2.0 network.

Rack design and coherence domains

Microsoft is using a new liquid-cooling design for Maia 300 racks and is moving from a 2U server form factor to 1U. This allows two dozen servers, each containing four Maia 300 accelerators, to fit in a rack. The result is 96 XPUs per rack.

Across 128 racks, the configuration can provide 12,288 accelerators in one system image. The resulting coherence domain is larger than those publicly described for other major cloud infrastructure systems. For comparison, Nvidia and AMD rack-scale systems have been described with coherent domains of 72 or 144 GPUs, as well as larger multilevel NVSwitch fabrics of 576 or 1,152 devices. Google’s “Sunfish” TPU 8t training chip, which is not yet shipping, is designed for up to 9,600 TPUs in a coherence domain.

Google’s “Zebrafish” TPU 8i inference chip uses a “boardfly” topology that reaches 32 devices in a boardfly. This consists of eight dragonfly nodes with four TPUs per node. Thirty-six boardflies can be linked through Google’s “Apollo” optical circuit switch, producing a group of 1,152 TPUs.

Reported performance

Microsoft’s published specifications describe raw system capabilities, which do not necessarily represent application performance. Rani Borkar, president of Azure Hardware Systems and Infrastructure, and Saurabh Dighe, corporate vice president of Systems Planning and Architecture, stated that a Maia 300 rack with 96 XPUs delivers up to seven times the token throughput of a Maia 200 rack with 48 XPUs.

The transition from Maia 200 to Maia 300 provides 36 percent more floating-point capability per XPU. Doubling the number of XPUs in a rack provides another 100 percent increase in accelerator count.

The reported improvement therefore depends on more than compute capacity. Maia 300 combines changes to accelerator coupling, HBM and SRAM capacity, memory bandwidth, network bandwidth, and dataflow. These changes are intended to move information through the system more efficiently during inference.

Microsoft reports that the sevenfold token-throughput improvement comes with a 2.5-fold improvement in performance per watt. The result reflects both higher rack-level performance and greater power efficiency.