An Infinite Replicated Softmax Model for Topic Modeling

In this paper, we describe the infinite replicated Softmax model (iRSM) as an adaptive topic model, utilizing the combination of the infinite restricted Boltzmann machine (iRBM) and the replicated Softmax model (RSM). In our approach, the iRBM extends the RBM by enabling its hidden layer to adapt to...

Full description

Saved in:
Bibliographic Details
Published inModeling Decisions for Artificial Intelligence pp. 307 - 318
Main Authors Huhnstock, Nikolas Alexander, Karlsson, Alexander, Riveiro, Maria, Steinhauer, H. Joe
Format Book Chapter Conference Proceeding
LanguageEnglish
Published Cham Springer International Publishing 2019
SeriesLecture Notes in Computer Science
Subjects
Online AccessGet full text

Cover

Loading…
More Information
Summary:In this paper, we describe the infinite replicated Softmax model (iRSM) as an adaptive topic model, utilizing the combination of the infinite restricted Boltzmann machine (iRBM) and the replicated Softmax model (RSM). In our approach, the iRBM extends the RBM by enabling its hidden layer to adapt to the data at hand, while the RSM allows for modeling low-dimensional latent semantic representation from a corpus. The combination of the two results is a method that is able to self-adapt to the number of topics within the document corpus and hence, renders manual identification of the correct number of topics superfluous. We propose a hybrid training approach to effectively improve the performance of the iRSM. An empirical evaluation is performed on a standard data set and the results are compared to the results of a baseline topic model. The results show that the iRSM adapts its hidden layer size to the data and when trained in the proposed hybrid manner outperforms the base RSM model.
ISBN:9783030267728
3030267725
9783030267735
3030267733
ISSN:0302-9743
1611-3349
DOI:10.1007/978-3-030-26773-5_27