Feature Enhanced Capsule Networks for Robust Automatic Essay Scoring

• Published in: Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track Vol. 12979; pp. 365 - 380
• Main Authors: Sharma, Arushi, Kabra, Anubha, Kapoor, Rajiv
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Adversarial testing; Automatic scoring; BERT; Capsule Neural Networks; Machine learning
• ISBN: 9783030865160; 3030865169
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-86517-7_23

Automatic Essay Scoring (AES) Engines have gained popularity amongst a multitude of institutions for scoring test-taker’s responses and therefore witnessed rising demand in recent times. However, several studies have demonstrated that the adversarial attacks severely hamper existing state-of-the-art AES Engines’ performance. As a result, we propose a robust architecture for AES systems that leverages Capsule Neural Networks, contextual BERT-based text representation, and key textually extracted features. This end-to-end pipeline captures semantics, coherence, and organizational structure along with fundamental rule-based features such as grammatical and spelling errors. The proposed method is validated by extensive experimentation and comparison with the state-of-the-art baseline models. Our results demonstrate that this approach performs significantly better on 6 out of 8 prompts on the Automated Student Assessment Prize (ASAP) dataset. In addition, it shows an overall best performance with a Quadratic Weighted Kappa (QWK) metric of 81%. Moreover, we empirically demonstrate that it is successful in identifying adversarial responses and scoring them lower.