INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledge

• Published in: arXiv.org
• Main Authors: Romanou, Angelika, outan, Negar, Sotnikova, Anna, Chen, Zeming, Nelaturu, Sree Harsha, Singh, Shivalika, Maheshwary, Rishabh, Altomare, Micol, Haggag, Mohamed A, Snegha, A, Amayuelas, Alfonso, Azril Hafizi Amirudin, Aryabumi, Viraat, Boiko, Danylo, Chang, Michael, Chim, Jenny, Cohen, Gal, Dalmia, Aditya Kumar, Diress, Abraham, Duwal, Sharad, Dzenhaliou, Daniil, Daniel Fernando Erazo Florez, Farestam, Fabian, Imperial, Joseph Marvin, Shayekh Bin Islam, Isotalo, Perttu, Jabbarishiviari, Maral, Karlsson, Börje F, Khalilov, Eldar, Klamm, Christopher, Koto, Fajri, Krzemiński, Dominik, de Melo, Gabriel Adriano, Montariol, Syrielle, Yiyang Nan, Niklaus, Joel, Novikova, Jekaterina, Johan Samir Obando Ceron, Debjit, Paul, Ploeger, Esther, Purbey, Jebish, Rajwal, Swati, Selvan Sunitha Ravi, Rydell, Sara, Roshan Santhosh, Sharma, Drishti, Skenduli, Marjana Prifti, Arshia Soltani Moakhar, Bardia Soltani Moakhar, Tamir, Ran, Tarun, Ayush Kumar, Wasi, Azmine Toushik, Weerasinghe, Thenuka Ovin, Yilmaz, Serhan, Zhang, Mike, Schlag, Imanol, Fadaee, Marzieh, Hooker, Sara, Bosselut, Antoine
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 29.11.2024
• Subjects: Benchmarks; English language; Generative artificial intelligence; Large language models; Non-English languages; Performance evaluation; Quality assessment; Regional development

The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal value of generative AI tools in many communities. However, the development of functional LLMs in many languages (\ie, multi...