Measuring clarity in legal text

• Published in: The University of Chicago law review Vol. 91; no. 1; pp. 1 - 82
• Main Author: Choi, Jonathan H
• Format: Journal Article
• Language: English
• Published: Chicago, IL University of Chicago Law School 01.01.2024; University of Chicago, acting on behalf of the University of Chicago Law Review
• Subjects: Artificial intelligence; Evidence; Indeterminacy (Linguistics); Judgments; Language and languages; Linguistics; Litigation; Meaning (Philosophy); UNITED STATES
• ISSN: 0041-9494; 1939-859X

Legal cases often turn on judgments of textual clarity: when the text is unclear, judges allow extrinsic evidence in contract disputes, consult legislative history in statutory interpretation, and more. Despite this, almost no empirical work considers the nature or prevalence of legal clarity. Scholars and judges who study real-world documents to inform the interpretation of legal text primarily treat unclear text as a research problem to be solved with more data rather than a fundamental feature of language. This Article makes both theoretical and empirical contributions to the legal concept of textual clarity. It first advances a theory of clarity that distinguishes between information and determinacy. A judge might find text unclear because she personally lacks sufficient information to decide which interpretation is best; alternatively, she might find it unclear because the text itself is fundamentally indeterminate. Fundamental linguistic indeterminacy explains ongoing interpretive debates and limits the potential for text-focused methods (including corpus linguistics) to decide cases. With this theoretical background, the Article then proposes a new method to algorithmically evaluate textual clarity. Applying techniques from natural language processing and artificial intelligence that measure the semantic similarity between words, we can shed valuable new light on questions of legal interpretation. This Article finds that text is frequently indeterminate in real-world legal cases. Moreover, estimates of similarity vary substantially from corpus to corpus, even for large and reputable corpora. This suggests that word use is highly corpus-pecific and that meaning can vary even between general-purpose corpora that theoretically capture ordinary meaning. These empirical findings have important implications for ongoing doctrinal debates, suggesting that text is less clear and objective than many textualists believe. Ultimately, the Article offers new insights both to theorists considering the role of legal text and to empiricists seeking to understand how text is used in the real world.