Collecting event‐related tweets from twitter stream

• Published in: Journal of the American Society for Information Science and Technology Vol. 70; no. 2; p. 176
• Main Authors: Zheng, Xin, Sun, Aixin
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Periodicals Inc 01.02.2019
• Subjects: Active Learning; Annotations; Disasters; Information flow; Information Retrieval; Information Transfer; Keywords; Learning; Natural disasters; Noise; Social networks; Space; Traffic flow
• ISSN: 2330-1635; 2330-1643
• DOI: 10.1002/asi.24096

Twitter provides a channel of collecting and publishing instant information on major events like natural disasters. However, information flow on Twitter is of great volume. For a specific event, messages collected from the Twitter Stream based on either location constraint or predefined keywords would contain a lot of noise. In this article, we propose a method to achieve both high‐precision and high‐recall in collecting event‐related tweets. Our method involves an automatic keyword generation component, and an event‐related tweet identification component. For keyword generation, we consider three properties of candidate keywords, namely relevance, coverage, and evolvement. The keyword updating mechanism enables our method to track the main topics of tweets along event development. To minimize annotation effort in identifying event‐related tweets, we adopt active learning and incorporate multiple‐instance learning which assigns labels to bags instead of instances (that is, individual tweets). Through experiments on two real‐world events, we demonstrate the superiority of our method against state‐of‐the‐art alternatives.