Breaking the Memory Wall: 333-eDRAM in Tensor Accelerators
A computer-architecture study of dense embedded DRAM for energy-efficient transformer accelerators
A computer-architecture study asking whether a new embedded DRAM — 333-eDRAM (IGZO + CNFET + CMOS) — can break the memory-wall bottleneck in a TPU v4-inspired tensor accelerator running GPT-style attention. Using AccelForge, I modeled four memory configurations (all-SRAM baseline, local-buffer eDRAM, global-buffer eDRAM, and all-eDRAM) and analyzed how energy and reuse change with sequence length and einsum fusion. The finding: 333-eDRAM pays off most when used selectively in the global buffer, where its density keeps large attention intermediates on-chip — cutting energy by up to 16.4% with near-flat latency.
Modeled four memory configurations of a TPU-style accelerator on GPT-style attention
Energy reductions of up to 16.4% from selective global-buffer 333-eDRAM
Showed gains concentrate at intermediate sequence lengths where SRAM begins to spill
Demonstrated einsum fusion amplifies eDRAM's benefit by keeping intermediates on-chip
The headline: where density helps
I swapped SRAM for 333-eDRAM in four configurations and measured energy. Putting eDRAM only in the local buffer actually hurt (the taller bar), but the global-buffer configuration cut energy the most — confirming the thesis that density helps exactly where large, reused tensors live, not everywhere.
Why it works: the spill point
Energy versus sequence length tells the why. As the GPT-style workload grows and attention intermediates spill out of SRAM, the 333-eDRAM design (orange) pulls steadily below the all-SRAM baseline (blue). The biggest separation appears at intermediate-to-long sequences, where extra on-chip capacity keeps QK and softmax intermediates from being re-fetched from main memory.
Controlling the dataflow
To make the comparison fair I had to control the dataflow. This is the AccelForge mapping tree for the baseline — how each tensor and einsum is tiled and staged across the local buffer, global buffer, and main memory of the TPU-style accelerator.
Fusion compounds the win
And this is the fused mapping. Once einsum fusion lets attention intermediates stay live in on-chip storage instead of being re-materialized in DRAM, 333-eDRAM's extra capacity compounds — which is why the energy gains are so sensitive to how the workload is mapped.
Want the full story?
Happy to talk through any of the engineering decisions, trade-offs, or what broke along the way.