Unbiased Learning for Hierarchical Models

It is known that overfitting occurs when a conventional statistical learning method such as maximum likelihood estimation, maximum a posteriori estimation or Bayesian estimation is applied to hierarchical models. This paper gives an explanation why overfitting occurs and propose an appropriate learn...

Full description

Saved in:
Bibliographic Details
Published in2007 International Joint Conference on Neural Networks pp. 575 - 580
Main Authors Sekino, M., Nitta, K.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.08.2007
Subjects
Application software
Bayesian methods
Computer simulation
Kernel
Linear regression
Maximum a posteriori estimation
Maximum likelihood estimation
Neural networks
Statistical learning
Testing
Online AccessGet full text

Cover

More Information
Summary:It is known that overfitting occurs when a conventional statistical learning method such as maximum likelihood estimation, maximum a posteriori estimation or Bayesian estimation is applied to hierarchical models. This paper gives an explanation why overfitting occurs and propose an appropriate learning framework unbiased learning for hierarchical models. The method suggest to train the hyperparameters based on unbiased likelihood which is estimated by an appropriate information criterion. Therefore, it can say that the unbiased learning is a generalization of hyperparameters selection. Unbiased learning with several information criteria is tested by computer simulations.
ISBN:9781424413799
1424413796
ISSN:2161-4393
2161-4407
DOI:10.1109/IJCNN.2007.4371020