Micron Technology, Inc. (NASDAQ:MU) announced it has begun shipping customer samples of what it calls the world’s first 256GB SOCAMM2 low-power DRAM module, marking a new capacity milestone for data center memory.
The module is powered by the industry’s first monolithic 32Gb LPDDR5X die design and is aimed at next-generation AI and high-performance computing (HPC) infrastructure. Micron said the breakthrough enables significantly higher memory capacity with lower power consumption, opening the door to new system architectures for AI-driven workloads.
As AI training, inference, agentic AI and traditional compute increasingly converge, memory requirements have grown more complex. Expanding model sizes, larger context windows and persistent key-value (KV) caches are placing pressure on capacity, bandwidth, latency and energy efficiency—factors that directly affect scalability and total cost of ownership in data centers.
Micron said LPDRAM’s blend of capacity, efficiency and performance makes it a foundational technology for both AI-focused and general-purpose servers operating in power- and thermally constrained environments. The company is collaborating with NVIDIA to co-design memory solutions tailored to advanced AI infrastructure.
“Micron’s 256GB SOCAMM2 offering enables the most power-efficient CPU-attached memory solution for both AI and HPC. Today’s announcement highlights Micron’s technology and packaging advancements to deliver the highest-capacity, lowest-power modular memory solution with the smallest footprint in the industry,” said Raj Narasimhan, senior vice president and general manager of Micron’s Cloud Memory Business Unit. “Our continued leadership in low-power memory solutions for data center applications has uniquely positioned us to be the first to deliver a 32Gb monolithic LPDRAM die, helping drive industry adoption of more power-efficient, high-capacity system architectures.”
Built for Capacity, Efficiency and AI Workloads
Micron outlined several performance advantages for the new 256GB SOCAMM2 module:
- Higher memory density: The module offers one-third more capacity than the previous 192GB SOCAMM2, enabling up to 2TB of LPDRAM per 8-channel CPU configuration. This is designed to support larger AI models and more complex inference tasks.
- Reduced power draw and footprint: SOCAMM2 consumes roughly one-third the power of comparable RDIMM solutions while occupying about one-third of the physical space, potentially improving rack density and lowering total cost of ownership.
- Faster AI inference: In unified memory architectures, Micron said the module can improve time to first token by more than 2.3 times for long-context, real-time large language model (LLM) inference when used for KV cache offload. In standalone CPU environments, LPDRAM delivers over three times better performance per watt compared with mainstream memory modules in HPC workloads.
- Modular scalability: The SOCAMM2 form factor enhances serviceability, supports liquid-cooled server designs and allows for future capacity expansion as AI and compute demands increase.
“Advanced AI infrastructure requires incredible optimization at every layer to maximize performance and efficiency for demanding AI reasoning workloads,” said Ian Finder, head of Product, Data Center CPUs at NVIDIA. “Micron’s achievements in delivering massive memory capacity and bandwidth using less power than traditional server memory with 256GB SOCAMM2 is enabling the next generation of AI CPUs.”
Industry Collaboration and Standards Leadership
Micron said it continues to play a central role in shaping the JEDEC SOCAMM2 specification and is working closely with system designers to drive improvements in power efficiency and performance across future data center platforms.
The company now offers what it describes as the industry’s broadest LPDRAM portfolio for data centers, ranging from 8GB to 64GB components and 48GB to 256GB SOCAMM2 modules. Customer sampling of the 256GB version is currently underway.