Hierarchical Structures Induce Long-Range Dynamical Correlations in Written Texts

Thoughts and ideas are multidimensional and often concurrent, yet they can be expressed surprisingly well sequentially by the translation into language. This reduction of dimensions occurs naturally but requires memory and necessitates the existence of correlations, e.g., in written text. However, c...

Full description

Saved in:
Bibliographic Details
Published inProceedings of the National Academy of Sciences - PNAS Vol. 103; no. 21; pp. 7956 - 7961
Main Authors Alvarez-Lacalle, E., Dorow, B., Eckmann, J.-P., Moses, E.
Format Journal Article
LanguageEnglish
Published United States National Academy of Sciences 23.05.2006
Subjects
Autocorrelation
Connectivity
Correlations
Language
Mathematical vectors
Mathematics
Matrices
Memory
Models, Statistical
Nouns
Physical Sciences
Power laws
Reading comprehension
Semantics
Systems Analysis
Trajectories
Vector spaces
Vocabulary
Words
Online AccessGet full text

Cover

More Information
Summary:Thoughts and ideas are multidimensional and often concurrent, yet they can be expressed surprisingly well sequentially by the translation into language. This reduction of dimensions occurs naturally but requires memory and necessitates the existence of correlations, e.g., in written text. However, correlations in word appearance decay quickly, while previous observations of long-range correlations using random walk approaches yield little insight on memory or on semantic context. Instead, we study combinations of words that a reader is exposed to within a "window of attention," spanning about 100 words. We define a vector space of such word combinations by looking at words that co-occur within the window of attention, and analyze its structure. Singular value decomposition of the co-occurrence matrix identifies a basis whose vectors correspond to specific topics, or "concepts" that are relevant to the text. As the reader follows a text, the "vector of attention" traces out a trajectory of directions in this "concept space." We find that memory of the direction is retained over long times, forming power-law correlations. The appearance of power laws hints at the existence of an underlying hierarchical network. Indeed, imposing a hierarchy similar to that defined by volumes, chapters, paragraphs, etc. succeeds in creating correlations in a surrogate random text that are identical to those of the original text. We conclude that hierarchical structures in text serve to create long-range correlations, and use the reader's memory in reenacting some of the multidimensionality of the thoughts being expressed.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
Author contributions: E.A.-L., E.M., B.D., and J.-P.E. designed research; E.A.-L., E.M., B.D., and J.-P.E. performed research; E.A.-L., B.D., and J.-P.E. analyzed data; and E.A.-L., E.M., and J.-P.E. wrote the paper.
Edited by Nancy J. Kopell, Boston University, Boston, MA, and approved March 27, 2006
ISSN:0027-8424
1091-6490
DOI:10.1073/pnas.0510673103