Native vs Non-Native Language Prompting: A Comparative Analysis
Large language models (LLMs) have shown remarkable abilities in different fields, including standard Natural Language Processing (NLP) tasks. To elicit knowledge from LLMs, prompts play a key role, consisting of natural language instructions. Most open and closed source LLMs are trained on available...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
11.09.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Large language models (LLMs) have shown remarkable abilities in different
fields, including standard Natural Language Processing (NLP) tasks. To elicit
knowledge from LLMs, prompts play a key role, consisting of natural language
instructions. Most open and closed source LLMs are trained on available labeled
and unlabeled resources--digital content such as text, images, audio, and
videos. Hence, these models have better knowledge for high-resourced languages
but struggle with low-resourced languages. Since prompts play a crucial role in
understanding their capabilities, the language used for prompts remains an
important research question. Although there has been significant research in
this area, it is still limited, and less has been explored for medium to
low-resourced languages. In this study, we investigate different prompting
strategies (native vs. non-native) on 11 different NLP tasks associated with 12
different Arabic datasets (9.7K data points). In total, we conducted 197
experiments involving 3 LLMs, 12 datasets, and 3 prompting strategies. Our
findings suggest that, on average, the non-native prompt performs the best,
followed by mixed and native prompts. |
---|---|
DOI: | 10.48550/arxiv.2409.07054 |