Can GPT-4 Chat Pass a Polish Stockbroker Exam?

• Published in: Zeszyty Naukowe Wyższej Szkoły Finansów i Prawa w Bielsku-Białej Vol. 28; no. 1
• Main Author: Tomasz Wyłuda
• Format: Journal Article
• Language: English
• Published: University of Applied Sciences in Bielsko-Biała 01.03.2024
• Subjects: Artificial Intelligence (AI); finance; stocks; the law
• ISSN: 2543-9103; 2543-411X
• DOI: 10.19192/wsfip.sj1.2024.10

This research investigates the performance of OpenAI's GPT-4, a sophisticated language model, in passing the Polish Stockbroker Exam conducted by the Polish Financial Supervisory Authority (KNF). The exam, covering a broad range of topics, including legal issues, finance theory, finance mathematics, and setting prices, requires theoretical and practical skills pertinent to the financial markets. The study is set against various evaluations where GPT-4 and its predecessors have been tested in numerous academic and professional settings, demonstrating strengths and weaknesses in different domains. The study aimed to determine whether GPT-4 can pass the Polish Stockbroker Exam and analyze its performance across different question types. Results indicated that GPT-4 consistently failed to meet the passing score. However, it performed better when given more time per question, suggesting a trade-off between accuracy and completeness. Analysis by question type revealed higher proficiency in legal and finance theoretical questions but significant struggles with specific questions related to the stockbroker job. Notably, GPT-4 showed improvement in finance calculation questions with more response time.