An information-theoretic perspective of tf–idf measures

• Published in: Information processing & management Vol. 39; no. 1; pp. 45 - 65
• Main Author: Aizawa, Akiko
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 2003; Elsevier Science; Elsevier Science Ltd
• Subjects: Document Handling; Exact sciences and technology; Frequency Data; Information and communication sciences; Information management; Information Retrieval; Information retrieval systems; Information retrieval systems. Information and document management system; Information science. Documentation; Information theory; Information theory approach; Mathematical Formulas; Measurement Techniques; Online information retrieval; Probability; Probability weighted amount of information; Sciences and techniques of general use; Searching; Specificity; Studies; Subject Index Terms; Term weighting theories; Text categorization; tf–idf; Theoretical Analysis; Theory; Weighted Data; Text categorization; Term weighting theories; tf–idf; Information theory; Word; Weighting; Measurement indicator; Occurrence frequency; Search system; Probability; Information retrieval; Algorithm; Categorization; Statistics
• ISSN: 0306-4573; 1873-5371
• DOI: 10.1016/S0306-4573(02)00021-3

This paper presents a mathematical definition of the “probability-weighted amount of information” (PWI), a measure of specificity of terms in documents that is based on an information-theoretic view of retrieval events. The proposed PWI is expressed as a product of the occurrence probabilities of terms and their amounts of information, and corresponds well with the conventional term frequency–inverse document frequency measures that are commonly used in today’s information retrieval systems. The mathematical definition of the PWI is shown, together with some illustrative examples of the calculation.