Boltzmann Machine Learning with the Latent Maximum Entropy Principle

We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME pr...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Wang, Shaojun, Schuurmans, Dale, Peng, Fuchun, Zhao, Yunxin
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 19.10.2012
Subjects
Algorithms
Economic models
Entropy
Estimating techniques
Machine learning
Maximum entropy
Maximum likelihood estimation
Parameter estimation
Statistical inference
Online AccessGet full text

Cover

More Information
Summary:We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving new algorithms for Boltzmann machine parameter estimation, and show how robust and fast new variant of the EM algorithm can be developed.Our experiments show that estimation based on LME generally yields better results than maximum likelihood estimation, particularly when inferring hidden units from small amounts of data.
ISSN:2331-8422