Harmony search for hyperparameters optimization of a low resource language transformer model trained with a novel parallel corpus Ocelotl Nahuatl – Spanish

• Published in: Systems and soft computing Vol. 6; p. 200152
• Main Authors: Pacheco Martínez, Máximo Enrique, Carrillo Ruiz, Maya, Sandoval Solís, María de Lourdes
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.12.2024; Elsevier
• Subjects: Harmony search; Low-resource language; Metaheuristics; Nahuatl; Neural machine translation; Parallel corpus; Soft computing; Spanish; Transformer models; Parallel corpus; Low-resource language; Transformer models; Harmony search; Nahuatl; Metaheuristics; Neural machine translation; Spanish; Soft computing
• ISSN: 2772-9419; 2772-9419
• DOI: 10.1016/j.sasc.2024.200152

Nahuatl, a low-resource language, does not have an online translator application. Instead, resources are limited to dictionaries, web pages, or digital books. Given this condition, it is vital to provide as much support to the language as possible. This research aims to enhance the BLEU score in machine translation by applying the harmony search heuristic method to state-of-the-art transformers models. This is conducted by finding the optimal hyperparameter settings for the models. Models are trained and tested using a fresh moderate-size parallel corpus of 1.5k phrases. By utilizing harmony search, the study shows an improvement in the BLEU score, enhancing it by 2.569%. In order to accomplish this, various factors related to the hyperparameters need to be considered. The application of harmony search with transformers can be extended to various parallel corpora or models, taking these considerations into account.