How DeepSeek-R1 was created?

• Published in: Shenzhen da xue xue bao. Li gong ban Vol. 42; no. 2; pp. 226 - 232
• Main Author: ZHANG Huimin
• Format: Journal Article
• Language: English
• Published: Science Press (China Science Publishing & Media Ltd.) 01.03.2025
• Subjects: artificial intelligence; deepseek; group relative policy optimization; large language model; mixed-precision training; mixture of experts architecture; multi-head latent attention mechanism; multi-token prediction

This article summarizes the innovations and optimizations in DeepSeek series models for large-scale training. The breakthroughs of DeepSeek are primarily reflected in model and algorithm innovations, software and hardware collaborative optimization, and the improvement of overall training efficiency...