Qualcomm Advances HBC Architecture for AI Inference Datacenters

Qualcomm Advances HBC Architecture for AI Inference Datacenters

Qualcomm has validated first-generation silicon for its high-bandwidth computing (HBC) architecture, a near-memory computing design intended to address the memory bottleneck affecting AI inference and agentic AI workloads.

The underlying issue, often called the memory wall, is that AI performance increasingly depends not only on a processor's ability to calculate, but also on how quickly data can reach the compute units. This affects both model performance and the cost of operating AI workloads.

Moving compute closer to memory

Qualcomm has outlined three approaches to the problem. One keeps the AI model in fast on-chip memory. Another uses stacked high-bandwidth memory (HBM) connected to a processor. Qualcomm's HBC approach places compute closer to the memory that stores the data.

According to Gerardo Giaretta, Qualcomm's vice president and general manager of datacenter CPU and AI, conventional systems typically use DDR memory with a relatively narrow data link. HBM increases the number of connections and the speed between memory and compute, but it also adds cost, power consumption, and packaging complexity through technologies such as interposers and 2.5D packaging.

HBC instead places an LPDDR memory stack above a compute die. This shortens the distance that data travels from millimeters to micrometers, increasing bandwidth and reducing the energy required to move data.

In the first generation, Qualcomm places a full AI accelerator beneath the LPDDR stack. Most AI operations run inside the HBC package, although some tasks can move to the central system-on-chip (SoC), Qualcomm's Dragonfly AI250, depending on the system architecture and implementation.

Relationship between HBC and AI250

Qualcomm has begun production of the AI250 inference platform, which is scheduled for release next year. The AI250 provides substantially more TFLOPs, while its interface with LPDDR is slower than the connection within the HBC package.

Giaretta described HBC as a co-processor that works alongside the AI250. Once the HBC is instructed, most decode operations take place within the package. Only results need to travel over the slower link, rather than the full data set, so the link does not significantly affect end-to-end performance.

Silicon validation results

Qualcomm used industry-standard automated test equipment (ATE) to validate its first-generation HBC silicon. The tests included the complete memory read-write path and near-memory computations at production levels.

The company reported that HBC delivered:

• Up to 18 times the memory bandwidth of LPDDR
• Five times the terabits-per-second-per-watt energy efficiency of HBM4

Qualcomm said the results demonstrate that HBC can provide the bandwidth required for inference while reducing the energy used to move data. The company characterized the reduction in data-movement energy as at least an order of magnitude from an operating-expenditure perspective.

Dragonfly roadmap

Qualcomm plans to integrate HBC into the Dragonfly AI250, creating what it describes as a complete AI inference accelerator platform. The AI250 is intended to form the foundation of the Qualcomm Dragonfly Rack, the company's infrastructure stack for inference workloads. Sampling is expected to begin in the first half of 2027.

The HBC validation is the first stage of a longer product roadmap. Samples of the Dragonfly AI250 and Dragonfly C1000 CPUs are planned for 2027, followed by samples of the Dragonfly AI300 in 2028.

Giaretta said HBC is intended to differentiate Qualcomm's approach from designs that focus only on increasing memory bandwidth. The architecture combines memory with dedicated compute beneath the memory stack, allowing the system to process data closer to where it is stored.

Qualcomm is also working with partners to explore additional uses for HBC beyond the Dragonfly AI250. The company said the silicon validation provides evidence supporting its plans for high-yield AI250 manufacturing next year, while acknowledging that the design represents a new type of chip integration for Qualcomm.