Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia

• Published in: arXiv.org
• Main Authors: Zhou, Zhejian, Wang, Jiayu, Lin, Dahua, Chen, Kai
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 27.09.2024
• Subjects: Large language models; Multiplication; Number systems

Though Large Language Models (LLMs) have shown remarkable abilities in mathematics reasoning, they are still struggling with performing numeric operations accurately, such as addition and multiplication. Numbers can be tokenized into tokens in various ways by different LLMs and affect the numeric op...