Implementing Domain-Specific Languages for Heterogeneous Parallel Computing
HyoukJoong Lee, Brown, Kevin J., Sujeeth, Arvind K., Chafi, H., Olukotun, K., Rompf, Tiark, Odersky, Martin
Published in IEEE MICRO (01.09.2011)
Published in IEEE MICRO (01.09.2011)
Get full text
Journal Article
Locality-Aware Mapping of Nested Parallel Patterns on GPUs
Hyoukjoong Lee, Brown, Kevin J., Sujeeth, Arvind K., Rompf, Tiark, Olukotun, Kunle
Published in 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture (01.12.2014)
Published in 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture (01.12.2014)
Get full text
Conference Proceeding
Building-Blocks for Performance Oriented DSLs
Rompf, Tiark, Sujeeth, Arvind K., Lee, HyoukJoong, Brown, Kevin J., Chafi, Hassan, Odersky, Martin, Olukotun, Kunle
Published in Electronic proceedings in theoretical computer science (01.09.2011)
Published in Electronic proceedings in theoretical computer science (01.09.2011)
Get full text
Journal Article
Hardware system synthesis from Domain-Specific Languages
George, Nithin, HyoukJoong Lee, Novo, David, Rompf, Tiark, Brown, Kevin J., Sujeeth, Arvind K., Odersky, Martin, Olukotun, Kunle, Ienne, Paolo
Published in 2014 24th International Conference on Field Programmable Logic and Applications (FPL) (01.09.2014)
Published in 2014 24th International Conference on Field Programmable Logic and Applications (FPL) (01.09.2014)
Get full text
Conference Proceeding
Automatic support for multi-module parallelism from computational patterns
George, Nithin, HyoukJoong Lee, Novo, David, Owaida, Muhsen, Andrews, David, Olukotun, Kunle, Ienne, Paolo
Published in 2015 25th International Conference on Field Programmable Logic and Applications (FPL) (01.09.2015)
Published in 2015 25th International Conference on Field Programmable Logic and Applications (FPL) (01.09.2015)
Get full text
Conference Proceeding
Multi-codec variable length decoder design with configurable processor
Get full text
Conference Proceeding
Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training
Xu, Yuanzhong, Lee, HyoukJoong, Chen, Dehao, Choi, Hongjun, Hechtman, Blake, Wang, Shibo
Year of Publication 28.04.2020
Year of Publication 28.04.2020
Get full text
Journal Article
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Lepikhin, Dmitry, Lee, HyoukJoong, Xu, Yuanzhong, Chen, Dehao, Firat, Orhan, Huang, Yanping, Krikun, Maxim, Shazeer, Noam, Chen, Zhifeng
Year of Publication 30.06.2020
Year of Publication 30.06.2020
Get full text
Journal Article
GSPMD: General and Scalable Parallelization for ML Computation Graphs
Xu, Yuanzhong, Lee, HyoukJoong, Chen, Dehao, Hechtman, Blake, Huang, Yanping, Joshi, Rahul, Krikun, Maxim, Lepikhin, Dmitry, Ly, Andy, Maggioni, Marcello, Pang, Ruoming, Shazeer, Noam, Wang, Shibo, Wang, Tao, Wu, Yonghui, Chen, Zhifeng
Year of Publication 10.05.2021
Year of Publication 10.05.2021
Get full text
Journal Article
ATTENTION NEURAL NETWORKS WITH CONDITIONAL COMPUTATION
Lepikhin, Dmitry, Krikun, Maxim, Xu, Yuanzhong, Huang, Yanping, Shazeer, Noam M, Firat, Orhan, Lee, HyoukJoong, Chen, Dehao, Chen, Zhifeng
Year of Publication 13.07.2023
Get full text
Year of Publication 13.07.2023
Patent
TRAINING GIANT NEURAL NETWORKS USING PIPELINE PARALLELISM
Ngiam, Jiquan, Huang, Yanping, Cheng, Youlong, Lee, HyoukJoong, Chen, Dehao, Chen, Zhifeng
Year of Publication 21.04.2022
Get full text
Year of Publication 21.04.2022
Patent
Exploring the limits of Concurrency in ML Training on Google TPUs
Kumar, Sameer, Bradbury, James, Young, Cliff, Wang, Yu Emma, Levskaya, Anselm, Hechtman, Blake, Chen, Dehao, Lee, HyoukJoong, Deveci, Mehmet, Kumar, Naveen, Kanwar, Pankaj, Wang, Shibo, Wanderman-Milne, Skye, Lacy, Steve, Wang, Tao, Oguntebi, Tayo, Zu, Yazhou, Xu, Yuanzhong, Swing, Andy
Year of Publication 06.11.2020
Year of Publication 06.11.2020
Get full text
Journal Article
Training giant neural networks using pipeline parallelism
Ngiam, Jiquan, Huang, Yanping, Cheng, Youlong, Lee, HyoukJoong, Chen, Dehao, Chen, Zhifeng
Year of Publication 25.01.2022
Get full text
Year of Publication 25.01.2022
Patent
Scale MLPerf-0.6 models on Google TPU-v3 Pods
Kumar, Sameer, Bitorff, Victor, Chen, Dehao, Chou, Chiachen, Hechtman, Blake, Lee, HyoukJoong, Kumar, Naveen, Mattson, Peter, Wang, Shibo, Wang, Tao, Xu, Yuanzhong, Zhou, Zongwei
Year of Publication 20.09.2019
Year of Publication 20.09.2019
Get full text
Journal Article
ATTENTION NEURAL NETWORKS WITH CONDITIONAL COMPUTATION
LEPIKHIN, Dmitry, LEE, HyoukJoong, KRIKUN, Maxim, XU, Yuanzhong, HUANG, Yanping, CHEN, Zhifeng, CHEN, Dehao, FIRAT, Orhan, SHAZEER, Noam M
Year of Publication 25.01.2023
Get full text
Year of Publication 25.01.2023
Patent
TRAINING GIANT NEURAL NETWORKS USING PIPELINE PARALLELISM
Ngiam, Jiquan, Huang, Yanping, Cheng, Youlong, Lee, HyoukJoong, Chen, Dehao, Chen, Zhifeng
Year of Publication 11.02.2021
Get full text
Year of Publication 11.02.2021
Patent
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
Huang, Yanping, Cheng, Youlong, Bapna, Ankur, Firat, Orhan, Chen, Mia Xu, Chen, Dehao, Lee, HyoukJoong, Ngiam, Jiquan, Le, Quoc V, Wu, Yonghui, Chen, Zhifeng
Year of Publication 16.11.2018
Year of Publication 16.11.2018
Get full text
Journal Article
Mesh-TensorFlow: Deep Learning for Supercomputers
Shazeer, Noam, Cheng, Youlong, Parmar, Niki, Tran, Dustin, Vaswani, Ashish, Koanantakool, Penporn, Hawkins, Peter, Lee, HyoukJoong, Hong, Mingsheng, Young, Cliff, Sepassi, Ryan, Hechtman, Blake
Year of Publication 05.11.2018
Year of Publication 05.11.2018
Get full text
Journal Article
Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training
Xu, Yuanzhong, Lee, HyoukJoong, Chen, Dehao, Choi, Hongjun, Hechtman, Blake, Wang, Shibo
Published in arXiv.org (28.04.2020)
Get full text
Published in arXiv.org (28.04.2020)
Paper
ATTENTION NEURAL NETWORKS WITH CONDITIONAL COMPUTATION
LEPIKHIN, Dmitry, LEE, HyoukJoong, KRIKUN, Maxim, XU, Yuanzhong, HUANG, Yanping, CHEN, Zhifeng, CHEN, Dehao, FIRAT, Orhan, SHAZEER, Noam M
Year of Publication 06.01.2022
Get full text
Year of Publication 06.01.2022
Patent