333-eDRAM as accelerator buffer memory

Why the buffer memory matters

A systolic accelerator is cheap at the multiply and expensive everywhere else. Every time a weight or an intermediate has to leave the chip and come back, it costs far more energy than the arithmetic done on it, so the size of the on-chip buffers decides a lot of the energy bill. SRAM is the default buffer memory because it’s fast and needs no refresh, but its six-transistor cell makes capacity expensive in area.

333-eDRAM (Kong et al., DAC 2025) is a three-transistor gain cell that stacks three transistor types: an IGZO write transistor whose very low off-state leakage gives long retention, a carbon-nanotube read transistor for drive current, and silicon CMOS underneath for the periphery. The question for an accelerator is whether its density buys enough extra capacity, in the same area, to cut off-chip traffic by more than its slower and costlier accesses give back.

What we modeled

The accelerator is AccelForge’s TPU v4 description: a 128 MiB global buffer feeding four local buffers, each in front of a 128 × 128 array of 8-bit MACs at 1.05 GHz, with DRAM as main memory. The SRAM numbers come with that description, which is based on the TPU v4 paper. We kept the area budget fixed and filled it with 333-eDRAM in the global buffer, the local buffers, or both:

The workload is one GPT-3 6.7B transformer block (query, key and value projections, QK, softmax, AV, the output projection and both feed-forward layers) at batch size 1 with no KV cache, swept from 64 to 2,048 tokens. AccelForge’s mapper picks the tiling and fusion for each configuration and reports energy and latency.

My part was getting 333-eDRAM priced at all. AccelForge asks a plug-in for each memory’s per-access energy, latency, leakage and area. I first approximated 333-eDRAM by scaling an existing eDRAM model with the cell numbers from the DAC paper, then replaced that with MemSysExplorer’s array characterizer (msxac) running the 333 cell, wrapped as an hwcomponents plug-in so AccelForge could call it like any other memory.

What the course model said

At 1,024 tokens:

Configuration	Energy vs. control	EDP vs. control
333-eDRAM local buffers	+14.3 %	+14.3 %
333-eDRAM global buffer	−12.5 %	−10.6 %
Both	+1.5 %	+3.7 %

Across sequence lengths the global-buffer design looked best at 1,280 tokens (−16.4 %) and then lost almost all of its advantage by 2,048. Turning off einsum fusion cost the 333 design 6.2 % and the SRAM design 3.0 %, which we read as fusion and capacity reinforcing each other. That was the paper’s story: use the dense memory selectively, where large tensors are reused.

What changed when I rebuilt the models

Over the summer I went back to extend the study to other accelerators and started with a sanity check: run the comparison on a workload where neither proposed mechanism can act, and see whether the tooling still reports a difference. It did. Tracing it back found problems in the memory-model chain rather than in AccelForge.

The density ratio was comparing different things

The 4× capacity in the same area came from a 4.14× area-per-bit ratio. That ratio compared a real SRAM macro from the TPU v4 paper, with all its periphery, banking and redundancy, against a tool-modeled eDRAM array that sits almost at bare cell area. Putting SRAM through the same characterizer, with the same node, capacity, width and wire model, changes the answer:

SRAM baseline at 128 MB	333-eDRAM density advantage
TPU v4 SRAM macro (literature)	4.14×
SRAM through CACTI	1.26×
SRAM through msxac (same tool as the eDRAM)	1.38×

The same-tool number lands almost exactly on the raw cell-area ratio (551 F² against 387.76 F², about 1.42×), which is the check that the tool’s overheads cancel. With a planar density advantage of about 1.4×, the same area budget no longer buys a meaningfully larger buffer, and the capacity argument mostly goes away. The monolithic-3D stacking that the DAC paper’s density claim rests on isn’t represented in the array model at all.

A null test that should show nothing

On a workload where neither proposed mechanism can act, pricing the baseline with the same-tool SRAM gives +1.45 %, which is the “nothing happened” answer it should give. Pricing it with the literature macro gives −20.24 %. That offset is independent of the workload, which means a large part of the original “win” was there before any data moved.

Defects in the model chain

The eDRAM was characterized as a cache, so it carried a tag array a scratchpad doesn’t have.
Access latency had been divided by the number of mats (sub-arrays), so a 128 MB array appeared to answer in 6.2 ps instead of about 10.8 ns.
Refresh power was never read from the tool output (0.111 W at 128 MB, 0.887 W at 512 MB).
DESTINY, used for a secondary 1T1C eDRAM baseline, silently clamped the 7 nm node to 22 nm.
A failed characterization run produced a memory with zero energy, area and leakage instead of an error, which would win any comparison it entered.

All of these are fixed now, and the null control passes against the same-tool baseline.

Where the course model’s saving came from

The paper explained the global-buffer result as large attention intermediates staying on chip instead of spilling to DRAM. The per-component breakdown doesn’t show that. In the re-run at 1,024 tokens (the lowest-EDP mapping described below), the global buffer’s energy per compute fell from 1.08 × 10⁻¹³ J to 6.0 × 10⁻¹⁴ J, while main-memory energy per compute was slightly higher in the 333 design (5.60 × 10⁻¹⁴ J against 5.49 × 10⁻¹⁴ J), not lower. Per access, the 333-eDRAM in the paper’s Table I is more expensive than SRAM. What it has is a lower leakage figure: 1.74 W against 5.60 W. Over the 3.2 ms run that difference alone is about 12 mJ, more than the whole 9.8 mJ saving. The result rested on the leakage number, which turned out to be the least stable quantity in the characterization and was missing refresh entirely.

The mapping mattered too

Separately, the course runs used the first mapping AccelForge returned for each configuration. Choosing the lowest-EDP mapping instead removes the peak at 1,280 tokens and the collapse at 1,536, and the fusion effect shrinks to 3.16 % vs. 2.95 %. Both curves below still use the original memory models, so neither should be read as the real benefit of 333-eDRAM. The point of the figure is how much the conclusion depended on mapper choice alone.

Sequence length (tokens)	Course paper (first mapping returned)	Re-run (lowest-EDP mapping)
64	−7.29	—
96	−6.50	—
128	−6.34	−6.64
136	−6.31	−7.17
144	−6.51	—
160	−7.36	—
192	−8.26	—
256	−10.19	−9.61
512	−12.67	−12.57
768	−14.13	—
1,024	−12.52	−13.58
1,280	−16.39	−13.57
1,536	−4.07	−13.42
2,048	−0.35	—