Cross-Language Assessment of Mathematical Capability of ChatGPT

This paper presents an evaluation of the mathematical capability of ChatGPT across diverse languages like Hindi, Gujarati, and Marathi. ChatGPT, based on GPT-3.5 by OpenAI, has garnered significant attention for its natural language understanding and generation abilities. However, its performance in...

Full description

Saved in:
Bibliographic Details
Main Authors Sathe, Gargi, Shamraj, Aneesh, Surve, Aditya, Patil, Nahush, Saxena, Kumkum
Format Journal Article
LanguageEnglish
Published 18.05.2024
Subjects
Computer Science - Computation and Language
Computer Science - Learning
Online AccessGet full text

Cover

More Information
Summary:This paper presents an evaluation of the mathematical capability of ChatGPT across diverse languages like Hindi, Gujarati, and Marathi. ChatGPT, based on GPT-3.5 by OpenAI, has garnered significant attention for its natural language understanding and generation abilities. However, its performance in solving mathematical problems across multiple natural languages remains a comparatively unexplored area, especially in regional Indian languages. In this paper, we explore those capabilities as well as using chain-of-thought prompting to figure out if it increases the accuracy of responses as much as it does in the English language and provide insights into the current limitations.
DOI:10.48550/arxiv.2405.11264