Nvidia is reportedly drawing up an ambitious plan to dominate AI inference with its next-generation Feynman chips, expected to launch in 2028. According to the latest in-depth analyses, the "Team Green" giant may integrate Groq's language processing units (LPUs) directly into its architecture. Rather than placing them side by side on a single plane, these units will more likely be stacked as separate dies, an approach reminiscent of the highly successful X3D technology that rival AMD uses.
Nvidia's Chip Stacking Technology and AI Inference Strategy
At first glance, the recent intellectual property licensing deal between Nvidia and Groq covering LPUs might look like a modest development if you only consider the size of the acquisition or the headline revenue figures. Behind the handshake, however, lies a major ambition: Nvidia wants to take the lead in AI inference through the power of LPUs. The question the tech world cares about most right now is how Nvidia will integrate this technology. Based on the assessment of GPU expert AGF, it looks like the LPUs will be stacked on top of the next-generation Feynman GPUs using TSMC's hybrid bonding technology.
This expert believes the real-world implementation could closely resemble what AMD did with its X3D CPUs. Specifically, AMD used TSMC's SoIC hybrid bonding technology to place 3D V-Cache tiles on top of the main compute die. AGF argues that integrating SRAM as a monolithic die alongside the processor may not be the right move for the Feynman GPU. The reason is that SRAM scaling is running into physical limits, and trying to build it on leading-edge process nodes would waste premium silicon area, significantly raising the cost per unit of wafer area.
Instead, the more sensible solution is for Nvidia to stack the LPUs on top of the Feynman compute die. In this model, chips built on the A16 (1.6nm) process would serve as the main Feynman die, holding the critical compute blocks such as the tensor units and control logic. Meanwhile, separate LPU dies would hold large banks of SRAM memory. To connect these components, TSMC's hybrid bonding would play a pivotal role, creating a wide, far more power-efficient interface than leaving the memory off-package. Additionally, because the A16 process offers backside power delivery, the front side of the chip would be freed up for vertical SRAM connections, ensuring ultra-low-latency decode responses.
Thermal Challenges and CUDA Software Compatibility
Although the idea sounds technically sound and promises high performance, the approach also raises quite a few concerns, especially around thermal management. Stacking chip dies on a process that runs at high compute density is already a major cooling challenge. With LPUs built to sustain a continuous flow of data throughput, the heat they generate could create performance bottlenecks if it is not handled well.
More importantly, the implications at the software execution level would also grow considerably with this approach. LPUs typically rely on a fixed execution order, which creates a natural conflict between the determinism and the flexibility of the system. Even if Nvidia solves the hardware constraints, the main concern remains how its renowned CUDA software platform will behave in an LPU-style execution environment.
The reason is that LPUs require memory placement to be explicit and specific, whereas CUDA kernels are designed to abstract the hardware to make programming easier. Integrating SRAM into AI architectures this way will not be an easy task for "Team Green." It will take a feat of engineering to make sure the combined LPU and GPU environment is as well optimized as possible. Still, this may be a price Nvidia is willing to pay if it truly wants to lead and dominate inference in the coming AI era.
