We’re launching Lyria 3.5 in Google Flow Music, with advances across musicality, lyrics, vocals, and creative control
deepmind.google·6d ago
AI Engineering
Hacker News·Jun 29·5 min read
Sophon PFG-1: a monolithic-3D AI ASIC with 330 GB of on-die DRAM and no HBM
TL;DR
Existing AI hardware often relies on off-die High-Bandwidth Memory (HBM), which can limit performance. The Sophon PFG-1 is a monolithic-3D AI ASIC that integrates 330 GB of on-die DRAM, enabling efficient training and inference.
✦ Why It Matters
Engineers can leverage the Sophon PFG-1 for more efficient AI training and inference without relying on HBM.
Key Takeaways
- Consider using Sophon PFG-1 for AI applications.
- Evaluate performance improvements over traditional GPUs.
- Explore the benefits of on-die DRAM integration.
- Investigate Compute-In-Memory architectures for efficiency.
Full Summary
AI hardware typically faces bandwidth limitations due to reliance on off-die High-Bandwidth Memory (HBM), which can hinder performance in training and inference tasks. The Sophon PFG-1 is a monolithic-3D AI application-specific integrated circuit (ASIC) that integrates 330 GB of on-die DRAM, allowing for both training and inference on a single chip.
Built on a 28 nm silicon complementary metal-oxide-semiconductor (CMOS) base, it utilizes a 32-tier transition-metal dichalcogenide (TMD) architecture. This design enables Compute-In-Memory (CIM) operations, achieving 4,200 TFLOPS in FP8 and 2,100 TFLOPS in BF16 formats.
The PFG-1 can handle 80 billion parameters with significant activation headroom, outperforming NVIDIA and AMD GPUs by 2.7–3.1× in training throughput and 48–53× in inference throughput. This architecture provides 191–214× the weight bandwidth of HBM4 packages, addressing a critical gap in AI hardware performance.