Algorithms for Deterministic Incremental Dependency Parsing

• Published in: Computational linguistics - Association for Computational Linguistics Vol. 34; no. 4; pp. 513 - 553
• Main Author: Nivre, Joakim
• Format: Journal Article
• Language: English
• Published: One Rogers Street, Cambridge, MA 02142-1209, USA MIT Press 01.12.2008; MIT Press Journals, The
• Subjects: Algorithms; Ambiguity; Analysis; Computational linguistics; Computer and Information Sciences Computer Science; Computer Applications; Computer science; Data- och informationsvetenskap; Datalogi; Datavetenskap; Datorlingvistik; Formal languages and grammars; HUMANIORA och RELIGIONSVETENSKAP; HUMANITIES and RELIGION; Information technology; Informationsteknik; Languages; Languages and linguistics; Linguistic subjects; Linguistics; Lingvistikämnen; Mathematics and linguistics; Natural Language Processing; Parsing; Språkvetenskap; Studies; TECHNOLOGY; TEKNIKVETENSKAP; Formal grammar; Mathematical linguistics; Algorithm; Syntactic parsing; Assessment
• ISSN: 0891-2017; 1530-9312; 1530-9312
• DOI: 10.1162/coli.07-056-R1-07-027

Parsing algorithms that process the input from left to right and construct a single derivation have often been considered inadequate for natural language parsing because of the massive ambiguity typically found in natural language grammars. Nevertheless, it has been shown that such algorithms, combined with treebank-induced classifiers, can be used to build highly accurate disambiguating parsers, in particular for dependency-based syntactic representations. In this article, we first present a general framework for describing and analyzing algorithms for deterministic incremental dependency parsing, formalized as transition systems. We then describe and analyze two families of such algorithms: stack-based and list-based algorithms. In the former family, which is restricted to projective dependency structures, we describe an arc-eager and an arc-standard variant; in the latter family, we present a projective and a non-projective variant. For each of the four algorithms, we give proofs of correctness and complexity. In addition, we perform an experimental evaluation of all algorithms in combination with SVM classifiers for predicting the next parsing action, using data from thirteen languages. We show that all four algorithms give competitive accuracy, although the non-projective list-based algorithm generally outperforms the projective algorithms for languages with a non-negligible proportion of non-projective constructions. However, the projective algorithms often produce comparable results when combined with the technique known as pseudo-projective parsing. The linear time complexity of the stack-based algorithms gives them an advantage with respect to efficiency both in learning and in parsing, but the projective list-based algorithm turns out to be equally efficient in practice. Moreover, when the projective algorithms are used to implement pseudo-projective parsing, they sometimes become less efficient in parsing (but not in learning) than the non-projective list-based algorithm. Although most of the algorithms have been partially described in the literature before, this is the first comprehensive analysis and evaluation of the algorithms within a unified framework.