Optimizing Low-Resource Zero-Shot Event Argument Classification with Flash-Attention and Global Constraints Enhanced ALBERT Model

• Published in: International journal of advanced computer science & applications Vol. 15; no. 8
• Main Authors: Sun, Tongyue, Xiao, Jiayi
• Format: Journal Article
• Language: English
• Published: West Yorkshire Science and Information (SAI) Organization Limited 01.01.2024
• Subjects: Annotations; Automation; Classification; Computer science; Constraints; Datasets; Efficiency; Flash memory (computers); Knowledge; Large language models; Natural language processing; Semantics
• ISSN: 2158-107X; 2156-5570
• DOI: 10.14569/IJACSA.2024.0150802

Event Argument Classification (EAC) is an essential subtask of event extraction. Most previous supervised models rely on costly annotations, and reducing the demand for computa-tional and data resources in resource-constrained environments is a significant challenge within the field. We propose a Zero-Shot EAC model, ALBERT-F, which leverages the efficiency of the ALBERT architecture combined with the Flash-Attention mechanism. This novel integration aims to address the limita-tions of traditional EAC methods, which often require extensive manual annotations and significant computational resources. The ALBERT-F model simplifies the design by factorizing embedding parameters, while Flash-Attention enhances computational speed and reduces memory access overhead. With the addition of global constraints and prompting, ALBERT-F improves the generaliz-ability of the model to unseen events. Our experiments on the ACE dataset show that ALBERT-F outperforms the Zero-shot BERT baseline by achieving at least a 3.4% increase in F1 score. Moreover, the model demonstrates a substantial reduction in GPU memory consumption by 75.1% and processing time by 33.3%, underscoring its suitability for environments with constrained resources.