Automatic new topic identification using multiple linear regression

• Published in: Information processing & management Vol. 42; no. 4; pp. 934 - 950
• Main Author: Ozmutlu, Seda
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.07.2006; Elsevier Science; Elsevier Science Ltd
• Subjects: ANOVA; Evaluation; Exact sciences and technology; Information and communication sciences; Information management; Information processing; Information processing and retrieval; Information retrieval; Information retrieval. Man machine relationship; Information science. Documentation; Online information retrieval; Performance measures; Queries; Regression; Regression analysis; Research process. Evaluation; Sciences and techniques of general use; Search engine; Search engines; Studies; Topic identification; Transaction logs; Use statistics; Regression; Information retrieval; Topic identification; ANOVA; Search engine; Query; Linear regression; Identification; Change; Multiple regression; Subject; Automatic processing
• ISSN: 0306-4573; 1873-5371
• DOI: 10.1016/j.ipm.2005.10.002

The purpose of this study is to provide automatic new topic identification of search engine query logs, and estimate the effect of statistical characteristics of search engine queries on new topic identification. By applying multiple linear regression and multi-factor ANOVA on a sample data log from the Excite search engine, we demonstrated that the statistical characteristics of Web search queries, such as time interval, search pattern and position of a query in a user session, are effective on shifting to a new topic. Multiple linear regression is also a successful tool for estimating topic shifts and continuations. The findings of this study provide statistical proof for the relationship between the non-semantic characteristics of Web search queries and the occurrence of topic shifts and continuations.