A framework for traversing dense annotation lattices

• Published in: Language Resources and Evaluation Vol. 44; no. 3; pp. 183 - 203
• Main Authors: Boguraev, Branimir, Neff, Mary
• Format: Journal Article Conference Proceeding
• Language: English
• Published: Dordrecht Springer 01.09.2010; Springer Netherlands; Springer Nature B.V
• Subjects: Annotations; Applied linguistics; Architectural engineering; Architecture; Computational Linguistics; Computer Science; Devices; Directives; Engines; Grammar; Grammars; Information management; Information retrieval; Language and Literature; Lattice matching; Lattices; Linguistics; Matching; Mathematical analysis; Mathematical lattices; Navigation; Noun phrases; Pattern matching; Pattern recognition; Repositories; Social Sciences; Stores; Symbolism; Syntactics; Word processing; New York; Czech Republic; United States > US; Spain; st; Finite-state transduction; High density annotation repositories; Annotation lattices; Annotation-based analytics development; Corpus analysis; Pattern matching over annotations; Corpus annotation; AFST; Natural language processing; Computational linguistics; UIMA
• ISSN: 1574-020X; 1572-8412; 1574-0218
• DOI: 10.1007/s10579-010-9123-y

Pattern matching, or querying, over annotations is a general purpose paradigm for inspecting, navigating, mining, and transforming annotation repositories—the common representation basis for modern pipelined text processing architectures. The open-ended nature of these architectures and expressiveness of feature structure-based annotation schemes account for the natural tendency of such annotation repositories to become very dense, as multiple levels of analysis get encoded as layered annotations. This particular characteristic presents challenges for the design of a pattern matching framework capable of interpreting 'flat' patterns over arbitrarily dense annotation lattices. We present an approach where a finite state device applies (compiled) pattern grammars over what is, in effect, a linearized 'projection' of a particular route through the lattice. The route is derived by a mix of static grammar analysis and runtime interpretation of navigational directives within an extended grammar formalism; it selects just the annotations sequence appropriate for the patterns at hand. For expressive and efficient pattern matching in dense annotations stores, our implemented approach achieves a mix of lattice traversal and finite state scanning by exposing a language which, to its user, provides constructs for specifying sequential, structural, and configurational constraints among annotations.