Near real-time atrocity event coding

In recent years, mass atrocities, terrorism, and political unrest have caused much human suffering. Thousands of innocent lives have been lost to these events. With the help of advanced technologies, we can now dream of a tool that uses machine learning and natural language processing (NLP) techniqu...

Full description

Saved in:
Bibliographic Details
Published in2016 IEEE Conference on Intelligence and Security Informatics (ISI) pp. 139 - 144
Main Authors Solaimani, Mohiuddin, Salam, Sayeed, Mustafa, Ahmad M., Khan, Latifur, Brandt, Patrick T., Thuraisingham, Bhavani
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.09.2016
Subjects
Apache Spark
Atrocity event
Big Data
Data mining
Encoding
Event coding
Feature extraction
Incremental learning
Natural language processing
Real-time systems
Sparks
Stream mining
Testing
Training
Online AccessGet full text

Cover

More Information
Summary:In recent years, mass atrocities, terrorism, and political unrest have caused much human suffering. Thousands of innocent lives have been lost to these events. With the help of advanced technologies, we can now dream of a tool that uses machine learning and natural language processing (NLP) techniques to warn of such events. Detecting atrocities demands structured event data that contain metadata, with multiple fields and values (e.g. event date, victim, perpetrator). Traditionally, humans apply common sense and encode events from news stories but this process is slow, expensive, and ambiguous. To accelerate it, we use machine coding to generate an encoded event. In this paper, we develop a near-real-time supervised machine coding technique with an external knowledge base, WordNet, to generate a structured event. We design a Spark-based distributed framework with a web scraper to gather news reports periodically, process, and generate events. We use Spark to reduce the performance bottleneck while processing raw text news using CoreNLP.
DOI:10.1109/ISI.2016.7745457