Leveraging Inter-step Dependencies for Information Extraction from Procedural Task Instructions

• Published in: Text, Speech, and Dialogue Vol. 12848; pp. 341 - 353
• Main Authors: Nabizadeh, Nima, Wersing, Heiko, Kolossa, Dorothea
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Information extraction; Instructional text; Long-term dependencies; Procedural task
• ISBN: 303083526X; 9783030835262
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-83527-9_29

Written instructions are among the most prevalent means of transferring procedural knowledge. Hence, enabling computers to obtain information from textual instructions is crucial for future AI agents. Extracting information from a step of a multi-part instruction is usually performed by solely considering the semantic and syntactic information of the step itself. In procedural task instructions, however, there is a sequential dependency across entities throughout the entire task, which would be of value for optimal information extraction. However, conventional language models such as transformers have difficulties processing long text, i.e., the entire instruction text from the first step to the last one, since their scope of attention is limited to a relatively short chunk of text. As a result, the dependencies among the steps of a longer procedure are often overlooked. This paper suggests a BERT-GRU model for leveraging sequential dependencies among all steps in a procedure. We present experiments on annotated datasets of text instructions in two different domains, i.e., repairing electronics and cooking, showing our model’s advantage compared to standard transformer models. Moreover, we employ a sequence prediction model to show the correlation between the predictability of tags and the performance benefit achieved by leveraging inter-step dependencies.