Softmax function design optimization and hardware realization method and system

• Main Authors: SHAO QIMING, ZHANG ZHUOJIAN, WANG QIN, WANG SHAOJUN
• Format: Patent
• Language: Chinese; English
• Published: 08.01.2019
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; PHYSICS

A Softmax function can complete the conversion from scalar to probability, and is widely used in the output layer of depth neural network classifier. Nowadays, as an important application of deep learning, multi-classification problem has more and more classification categories and higher precision requirements. The invention provides a Softmax function design optimization and hardware realizationmethod and system. According to a large number of input data, wide input range and high precision requirements, the invention calculates through two input modes to reduce on-chip storage resources, responds to a plurality of input pointing schemes through configurable lookup table, and determines output pointing schemes through hardware to improve precision. Softmax函数可以完成标量到概率的转换，被广泛应用在深度神经网络分类器中的输出层。时下，多分类问题作为深度学习的重要应用有着分类类别越来越多，精度要求越来越高的应用趋势。本发明提出了种Softmax函数的设计优化及硬件实现方法及系统，针对大量输入数据个数、广输入范围与高精度要求，本发明通过两遍输入的输入模式进行计算以减少片上存储资源、通过可配置查找表以应对多种输入定点化方案、通过硬件决定输出定点化方案以提高精度。