Analysis of Posit and Bfloat Arithmetic of Real Numbers for Machine Learning

• Published in: IEEE access Vol. 9; pp. 82318 - 82324
• Main Authors: Romanov, Aleksandr Yu, Stempkovsky, Alexander L., Lariushkin, Ilia V., Novoselov, Georgy E., Solovyev, Roman A., Starykh, Vladimir A., Romanova, Irina I., Telpukhov, Dmitry V., Mkrtchan, Ilya A.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Artificial intelligence; benchmark; floating point; Floating point arithmetic; Hardware; IEEE 754; Libraries; Linear algebra; Machine learning; Machine learning algorithms; Memory management; Neural networks; posit; Real numbers; Software
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3086669

Modern computational tasks are often required to not only guarantee predefined accuracy, but get the result fast. Optimizing calculations using floating point numbers, as opposed to integers, is a non-trivial task. For this reason, there is a need to explore new ways to improve such operations. This paper presents analysis and comparison of various floating point formats - float, posit and bfloat. One of the promising areas in which the problem of using such values can be considered to be the most acute is neural networks. That is why we pay special attention to algorithms of linear algebra and artificial intelligence to assess efficiency of new data types in this area. The research results showed that software implementations of posit16 and posit32 have high accuracy, but they are not particularly fast; on the other hand, bfloat16 is not much different from float32 in accuracy, but significantly surpasses it in performance for large amounts of data and complex machine learning algorithms. Thus, posit16 can be used in systems with less stringent performance requirements, as well as in conditions of limited computer memory; and also in cases when bfloat16 cannot provide required accuracy. As for bfloat16, it can speed up systems based on the IEEE 754 standard, but it cannot solve all the problems of conventional floating point arithmetic. Thus, although posits and bfloats are not a full fledged replacement for float, they provide (under certain conditions) advantages that can be useful for implementation of machine learning algorithms.