Variational Bayesian inference and complexity control for stochastic block models

• Published in: Statistical modelling Vol. 12; no. 1; pp. 93 - 115
• Main Authors: Latouche, P, Birmelé, E, Ambroise, C
• Format: Journal Article
• Language: English
• Published: New Delhi, India SAGE Publications 01.02.2012; Sage Publications Ltd
• Subjects: Algorithms; Applications; Approximation; Cluster analysis; Clustering; Methodology; Optimization techniques; Social sciences; Software; Statistics; Stochastic models; Studies; variational Bayes EM; integrated observed-data likelihood; integrated complete-data likelihood; variational EM; random graphs; stochastic block models; community detection
• ISSN: 1471-082X; 1477-0342
• DOI: 10.1177/1471082X1001200105

It is now widely accepted that knowledge can be acquired from networks by clustering their vertices according to the connection profiles. Many methods have been proposed and in this paper we concentrate on the Stochastic Block Model (SBM). The clustering of vertices and the estimation of SBM model parameters have been subject to previous work, and numerous inference strategies such as variational expectation maximization (EM) and classification EM have been proposed. However, SBM still suffers from a lack of criteria to estimate the number of components in the mixture. To our knowledge, only one model-based criterion, Integrated Complete-data Likelihood (ICL), has been derived for SBM in the literature. It relies on an asymptotic approximation of the integrated complete-data likelihood and recent studies have shown that it tends to be too conservative in the case of small networks. To tackle this issue, we propose a new criterion that we call Integrated Likelihood Variational Bayes (ILvb), based on a non-asymptotic approximation of the marginal likelihood. We describe how the criterion can be computed through a variational Bayes EM algorithm.