Shrinkage with shrunken shoulders: Gibbs sampling shrinkage model posteriors with guaranteed convergence rates

• Published in: Bayesian analysis Vol. 18; no. 2; pp. 367 - 390
• Main Authors: Nishimura, Akihiko, Suchard, Marc A
• Format: Journal Article
• Language: English
• Published: United States 01.06.2023
• Subjects: generalized linear model; secondary 62J07; Markov chain Monte Carlo; sparsity; Bayesian inference; Primary 60J20, 62F15; ergodicity
• ISSN: 1936-0975; 1931-6690
• DOI: 10.1214/22-BA1308

Use of continuous shrinkage priors - with a "spike" near zero and heavy-tails towards infinity - is an increasingly popular approach to induce sparsity in parameter estimates. When the parameters are only weakly identified by the likelihood, however, the posterior may end up with tails as heavy as the prior, jeopardizing robustness of inference. A natural solution is to "shrink the shoulders" of a shrinkage prior by lightening up its tails beyond a reasonable parameter range, yielding a version of the prior. We develop a regularization approach which, unlike previous proposals, preserves computationally attractive structures of original shrinkage priors. We study theoretical properties of the Gibbs sampler on resulting posterior distributions, with emphasis on convergence rates of the Pólya-Gamma Gibbs sampler for sparse logistic regression. Our analysis shows that the proposed regularization leads to geometric ergodicity under a broad range of global-local shrinkage priors. Essentially, the only requirement is for the prior on the local scale to satisfy . If further satisfies for , as in the case of Bayesian bridge priors, we show the sampler to be uniformly ergodic.