Interpreting the prediction process of a deep network constructed from supervised topic models

• Published in: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 2429 - 2433
• Main Authors: Chen, Jianshu, He, Ji, He, Xiaodong, Xiao, Lin, Gao, Jianfeng, Deng, Li
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.03.2016
• Subjects: Analytical models; Architecture; back propagation; Boundaries; BP-sLDA; Computer architecture; Decomposition; deep architecture; Dirichlet problem; Electronics; Logistics; Mathematical models; mirror descent; Mirrors; Networks; Neural networks; Predictive models; Probabilistic logic; Tasks; Topic model
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP.2016.7472113

In this paper, we propose an approach to interpret the prediction process of the BP-sLDA model, which is a supervised Latent Dirichlet Allocation model trained by Back Propagation over a deep architecture. The model is shown to achieve state-of-the-art prediction performance on several large-scale text analysis tasks. To interpret the prediction process of the model, often demanded by business data analytics applications, we perform evidence analysis on each pair-wise decision boundary over the topic distribution space, which is decomposed into a positive and a negative components. Then, for each element in the current document, a novel evidence score is defined by exploiting this topic decomposition and the generative nature of LDA. Then the score is used to rank the relative evidence of each element for the effectiveness of model prediction. We demonstrate the effectiveness of the method on a large-scale binary classification task on a corporate proprietary dataset with business-centric applications.