A generalization of BIC for the general exponential family

• Published in: Journal of statistical planning and inference Vol. 136; no. 9; pp. 2847 - 2872
• Main Authors: Chakrabarti, Arijit, Ghosh, Jayanta K.
• Format: Journal Article
• Language: English
• Published: Lausanne Elsevier B.V 01.09.2006; New York,NY Elsevier Science; Amsterdam
• Subjects: AIC; Bayes factor; BIC; Exact sciences and technology; General topics; Laplace approximation; Mathematics; Numerical analysis; Numerical analysis. Scientific computation; Numerical approximation; Parametric inference; Posterior normality; Probability and statistics; Sciences and techniques of general use; Statistics; Bayes factor; AIC; BIC; Posterior normality; 62F15; Laplace approximation; Bayes estimation; Error estimation; Information criterion; Bayes factor; BIC; AIC; Posterior normality; Laplace approximation; Statistical estimation; Statistical decision; Numerical approximation; Statistical method; Approximation error; Exponential family
• ISSN: 0378-3758; 1873-1171
• DOI: 10.1016/j.jspi.2005.01.005

In a normal example of Stone (1979, J. Roy. Statist. Soc. Ser. B 41, 276–278), Berger et al. (2003, J. Statist. Plann. Inference 112, 241–258) showed BIC may be a poor approximation to the logarithm of Bayes Factor. They proposed a Generalized Bayes Information Criterion (GBIC) and a Laplace approximation to the log Bayes factor in that problem. We consider a fairly general case where one has p groups of observations coming from an arbitrary general exponential family with each group having a different parameter and r observations. We derive a GBIC and a Laplace approximation to the integrated likelihood, under the assumption that p → ∞ and r → ∞ (and some additional restrictions, which vary from example to example). The general derivation clarifies the structure of GBIC. A general theorem is presented to prove the accuracy of approximation, and the worst possible approximation error is derived for several examples. In several numerical examples, the Laplace approximation and GBIC are seen to be quite good. They perform much better than BIC.