Early-Stage Event Prediction for Longitudinal Data

• Published in: Advances in Knowledge Discovery and Data Mining pp. 139 - 151
• Main Authors: Fard, Mahtab J., Chawla, Sanjay, Reddy, Chandan K.
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing 2016
• Series: Lecture Notes in Computer Science
• Subjects: Longitudinal data; Prediction; Regression; Survival analysis
• ISBN: 3319317520; 9783319317526
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-31753-3_12

Predicting event occurrence at an early stage in longitudinal studies is an important problem which has high practical value. As opposed to the standard classification and regression problems where a domain expert can provide the labels for the data in a reasonably short period of time, training data in such longitudinal studies must be obtained only by waiting for the occurrence of sufficient number of events. The main objective of this work is to predict the event occurrence in the future for a particular subject in the study using the data collected at the initial stages of a longitudinal study. In this paper, we propose a novel Early Stage Prediction (ESP) framework for building event prediction models which are trained at early stages of longitudinal studies. More specifically, we develop two probabilistic algorithms based on Naive Bayes and Tree-Augmented Naive Bayes (TAN), called ESP-NB and ESP-TAN, respectively, for early stage event prediction by modifying the posterior probability of event occurrence using different extrapolations that are based on Weibull and Lognormal distributions. The proposed framework is evaluated using a wide range of synthetic and real-world benchmark datasets. Our extensive set of experiments show that the proposed ESP framework is able to more accurately predict future event occurrences using only a limited amount of training data compared to the other alternative approaches.