3.5D Advanced Packaging Enabling Heterogenous Integration of HPC and AI Accelerators

Exponential growth in the number of parameters used to train deep neural network (DNN)/machine learning (ML) models for artificial intelligence (AI) training/ inference applications requires extensive compute resources like CPUs, GPUs, and memory, interconnected at extremely high bandwidth. Heteroge...

Full description

Saved in:
Bibliographic Details
Published in2024 IEEE 74th Electronic Components and Technology Conference (ECTC) pp. 798 - 802
Main Authors Mandalapu, Chandra Sekhar, Buch, Chintan, Shah, Priyal, Topacio, Roden, Cheng, Patrick, Wang, Liwei, Swaminathan, Raja, Smith, Alan, Wuu, John, Mysore, Kaushik, Alam, Arsalan
Format Conference Proceeding
LanguageEnglish
Published IEEE 28.05.2024
Subjects
3.5D Packaging
Advanced packaging
Bandwidth
Cooling
Electronic components
High bandwidth memory (HBM)
Hybrid bonding
Learning (artificial intelligence)
Memory management
Metal TIM
Metals
MI300X Instinct
Multichip modules
Passives
Reliability
Silicon interposer
Online AccessGet full text

Cover

More Information
Summary:Exponential growth in the number of parameters used to train deep neural network (DNN)/machine learning (ML) models for artificial intelligence (AI) training/ inference applications requires extensive compute resources like CPUs, GPUs, and memory, interconnected at extremely high bandwidth. Heterogeneous integration via chiplet architectures is key to enabling economically feasible growth of power efficient computing, given the slowdown in Moore's law. In this paper, we summarize innovative advanced packaging technologies that directly enabled the heterogenous integration of multiple chiplets including CPUs, GPUs, IO die, high bandwidth memory (HBM) die, and passive components in the largest, most complex, and high power (750 W) MI300X Instinct™ accelerator package built by AMD. Three key technologies are described: direct Cu-Cu hybrid bonding, 2.5D integration on a large silicon interposer, and metal thermal interface (TIM)-based cooling solution. The resulting 3.5D packaging technology is described and package-level reliability results are presented.
ISSN:2377-5726
DOI:10.1109/ECTC51529.2024.00391