Tracking Topic Evolution in On-Line Postings: 2006 IBM Innovation Jam Data

• Published in: Advances in Knowledge Discovery and Data Mining pp. 616 - 625
• Main Authors: Kobayashi, Mei, Yung, Raylene
• Format: Book Chapter
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg
• Series: Lecture Notes in Computer Science
• Subjects: discussion mining; topic detection and tracking
• ISBN: 9783540681243; 3540681248
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-540-68125-0_57

Participants in on-line discussion forums and decision makers are interested in understanding real-time communications between large numbers of parties on the internet and intranet. As a first step towards addressing this challenge, we developed a prototype to quickly identify and track topics in large, dynamic data sets based on assignment of documents to time slices, fast approximation of cluster centroids to identify discussion topics, and inter-slice correspondence mappings of topics. To verify our method, we conducted implementation studies with data from Innovation Jam 2006, an on-line brainstorming session, in which participants around the globe posted more than 37,000 opinions. Results from our prototype are consistent with the text in the postings, and would have required considerable effort to discover manually.