A Proposed Architecture for Reducing Memory-Wall Bottlenecks in High-Performance Systems

Modern analytical workloads frequently process massive datasets stored in formats such as .parquet, column-oriented databases, and structures analyzed through geometric or matrix operations. Complex queries, table shuffles, and joins can leave advanced server processors operating below their theoretical peak because cores spend substantial time waiting for data from RAM.

The proposed architecture addresses this problem by replacing conventional linear memory organization with a Z-Order, or Morton Layout, address space. It combines spatially organized memory, distributed cache-coherence filtering, geometric range checks, and larger cache blocks to reduce interconnect traffic and improve locality.

The Memory-Wall Problem

Traditional processors treat memory as a one-dimensional array. With 128 cores running analytical workloads concurrently, many cores may request narrow 64-byte cache lines scattered throughout RAM. Maintaining cache coherency requires the processor to issue large numbers of snoop queries asking which core owns each cache line.

This control traffic can congest the internal interconnect. Instead of using its bandwidth primarily for data movement, the interconnect becomes occupied with coherence management, leaving compute cores idle while they wait for the required data.

The proposed solution begins with a different question: what would happen if physical RAM addresses were organized using a spatial Z-Order format rather than a strictly linear sequence?

A Z-Order Architecture Based on AMD Turin

The reference design uses the AMD "Turin" Epyc 9005 architecture as a theoretical baseline. This processor configuration includes 128 compute cores distributed symmetrically across 16 Core Complex (CCX) modules.

The design treats application data as a multidimensional structure, similar to a solved Rubik's Cube. Each face can represent a table column or a dimension in a simulation. Software may preserve relationships between neighboring values, but a conventional memory system flattens those relationships into a single linear sequence.

When software needs one region, such as a column represented by one face of the cube, the processor may need to gather values separated by large memory strides. The proposed architecture instead interleaves address bits so that related spatial regions remain close together in the memory hierarchy.

Preserving Spatial Locality in Hardware

With Z-Order addressing, the hardware maintains the multidimensional structure during memory placement. When a core requests one region of the data, the cache can fetch a larger spatial block containing nearby values.

Software loop interchange can already improve performance by changing the order in which matrix data is traversed. In this design, similar locality behavior becomes a native hardware property. Traversing by rows or columns would produce more consistent cache behavior because the address layout preserves neighboring regions regardless of traversal direction.

Treating Caches as Data Structures

An L3 cache is a collection of data structures implemented in silicon. Software systems processing large datasets commonly use geometric indexes and minimum and maximum metadata to skip irrelevant records. The proposed architecture applies comparable techniques directly to the L2 and L3 cache structures.

Rather than treating every lookup as a search through a large linear collection, the cache hierarchy would use spatial organization and range metadata to reject requests that cannot match the data held in a cache region.

Z-Order Address Conversion

The central change affects address organization rather than instruction execution. Z-Order conversion is placed in the Memory Management Unit (MMU) and Translation Lookaside Buffer (TLB), with a stated deterministic cost of one clock cycle, or 1T.

The L2 and L3 caches, along with the internal interconnect, operate on Z-Order addresses. When the processor communicates with system RAM, a dedicated hardware decoder converts the Z-Order address back into the conventional physical layout expected by memory controllers, also within a stated 1T window.

The proposed transformation of a standard 48-bit physical address works as follows:

1. Offset isolation: The lower 6 bits, bits 0 through 5, are left unchanged because they identify the position within a 64-byte cache line.
2. Window splitting: The remaining bits, beginning at bit 6, are divided into two equal windows. One contains the most significant bits, or MSB, and the other contains the least significant bits, or LSB.
3. Bit interleaving: Bits from the LSB window are placed in even positions of the new address, such as positions 0, 2, 4, and 6. Bits from the MSB window are placed in odd positions, such as positions 1, 3, 5, and 7.

This arrangement places data that is related by rows, columns, or other spatial dimensions closer together in the physical address space.

Different Roles for L1, L2, and L3

The proposed architecture keeps the private L1 and L2 caches close to each core and continues to use 64-byte cache-line accesses at the local level. This preserves fine-grained operation for individual core workloads.

At the larger scale, the 512 MB L3 cache uses 4 KB chunks as mini-pages. In a Z-Order layout, each chunk represents a dense spatial region rather than a long linear section of a RAM row. Loading such a block can capture horizontal and vertical neighbors in a matrix at the same time.

The design assumes that later iterations of an analytical algorithm are likely to reuse those nearby values. Keeping them in the L3 cache could therefore reduce additional memory requests.

Distributed NoC Bloom Filters

The architecture adds a 4-bit sliced Counting Bloom Filter to the routers of the internal Network-on-Chip (NoC). On an AMD Turin-class topology, the global structure is divided into 16 segments of 32 KB each, with one segment placed near each of the 16 CCXs. Together, the segments occupy 512 KB.

The 4-bit counters are designed to pack two indices into each byte. Lookup and decoding are specified to complete within a 1T to 2T window.

The filter tracks active pages at a coarse level. When it indicates that a requested page is not present in a particular region, the system can avoid broadcasting a coherence query throughout the processor and instead direct likely matches to the relevant CCX. Because Bloom filters can produce false positives, additional range checks are used later in the hierarchy.

Enlarged L2 Caches and the Tropf-Hertzog Engine

The design increases private L2 capacity to exactly 2 MB per core. A 2 MB SRAM array is expected to add approximately 2T to the native access path. This capacity matches the 2 MB Huge Page allocation window used by the Linux kernel.

The larger working set could reduce long-tail requests to outer cache levels, despite the additional local latency. The cache continues to use 64-byte access granularity.

To protect the expanded L2, the design uses eight boundary-level minimum and maximum range filters supported by hardware BIGMIN and LITMAX functions. These functions are based on the 1981 Tropf-Hertzog algorithm described in Multidimensional Range Search in Dynamically Balanced Trees, published by Vieweg Verlag in Wiesbaden, Germany.

Space-filling curves have a limitation: moving between quadrants can create large linear gaps. A conventional binary search may perform poorly when valid and invalid regions are interleaved along the curve.

The proposed range engine addresses this by identifying when traversal has left the current query window. Tropf-Hertzog logic calculates the LITMAX value for the end of the current valid region and the BIGMIN value for the beginning of the next useful region. The L2 can then skip the invalid interval rather than scanning it or sending additional snoops to neighboring cores.

The design claims that these filters can return a mathematically sound negative result in 95 percent of cases within a 1T window. This would reduce unnecessary tag-array activity, pipeline stalls, and memory-port use.

Dual-Barrier Ingestion and Power Management

For large streaming reads, including columnar Parquet data, the L3 cache receives 4 KB blocks from system RAM. The block size aligns with Linux pages and with boundaries used by NVMe flash and SSD storage systems.

A second group of eight minimum and maximum filters is placed at the L3 level. If a request passes through the NoC Bloom filters but falls outside the relevant L3 region, these filters can issue a Fast Reject within a stated 1T window, avoiding searches through a large 16 MB L3 slice.

The resulting hierarchy uses eight filter pairs at L2 and eight filter pairs at L3. These barriers are intended to keep the primary tag arrays from repeatedly handling invalid or out-of-bounds requests.

Reducing unnecessary tag-array activation could also reduce power consumption. In conventional designs, speculative lookups may activate large SRAM tag structures even when the requested data is absent. In the proposed system, the Counting Bloom filters and Tropf-Hertzog range logic screen requests before those arrays are fully activated. The arrays would be powered up only when earlier checks indicate a likely cache hit.

SmartNIC and DPU Offloading

SmartNICs can place incoming network traffic directly into system RAM using technologies such as Remote Direct Memory Access (RDMA). Maintaining coherency can require the network interface to query the CPU about cache state, adding snoop traffic while the processor is handling data-intensive workloads.

The proposed design gives the SmartNIC direct access to the boundary-level Tropf-Hertzog geometric shield. Instead of waking the processor hierarchy with a broad snoop request, the interface checks the shield within a stated 1T window.

The intended effects are:

• Reduced core interruptions: CPU cores would not need to manage as much network-coherency traffic.
• Higher network utilization: The SmartNIC would be less dependent on queue processing by an overloaded CPU.
• Lower NoC power use: Removing redundant control traffic would reduce activity across the internal network.

This approach is aimed at systems that process millions of network packets per second alongside large Big Data or AI workloads.

Inter-Socket and CXL Traffic

The 4 KB macro-block design also applies to transfers between processor sockets. When one socket requests data from another, the system pays the interconnect latency once to transfer a spatially dense block containing nearby coordinates and data points.

The combination of CCX-level Counting Bloom filters and L2 Tropf-Hertzog logic can reject some requests before they cross the motherboard. Inter-processor links would then carry more useful data and less invalid control traffic.

The same concept is extended to disaggregated memory accessed through Compute Express Link (CXL). CXL-connected memory introduces additional latency through copper or fiber-optic cabling. A Counting Bloom filter extended to CXL-mapped address spaces could inspect a request before it leaves the local chip.

If the filter indicates that a page is absent or uninitialized, the request can be terminated at the silicon boundary. The proposed design attributes approximately 250 nanoseconds of recovered tail latency to this early rejection. When a CXL access proceeds, it retrieves a complete 4 KB Z-Order mini-page, paying the physical-distance cost once for a larger spatial block.

Target Workloads

The proposed processor is specialized rather than a general-purpose design intended for arbitrary workloads. Its intended applications include:

• Seismic processing and oil and gas exploration
• Artificial intelligence and machine learning
• Distributed SQL analytical engines
• Scientific computing
• Advanced engineering simulations
• Aerospace, defense, and space technologies

These workloads often process large, structured datasets in which spatial locality and range filtering are important.

Conclusion

The architecture moves several software techniques into processor hardware. Z-Order addressing organizes memory around multidimensional locality, distributed Counting Bloom filters reduce unnecessary coherence traffic, Tropf-Hertzog range logic skips invalid address intervals, and 4 KB cache blocks provide larger spatial transfers.

Together, these mechanisms are intended to reduce interconnect congestion, limit unnecessary cache-array activity, lower power consumption, and provide data-intensive workloads with a cache hierarchy that reflects the geometric structure of their data.