Practical variable selection for generalized additive models

• Published in: Computational statistics & data analysis Vol. 55; no. 7; pp. 2372 - 2387
• Main Authors: Marra, Giampiero, Wood, Simon N.
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.07.2011; Elsevier
• Series: Computational Statistics & Data Analysis
• Subjects: Additives; Computer simulation; Cross sections; Data processing; Estimators; Exact sciences and technology; General topics; Generalized additive model; Generalized additive model Nonnegative garrote estimator Penalized thin plate regression spline Practical variable selection Shrinkage smoother; Guidelines; Linear inference, regression; Mathematical models; Mathematics; Multivariate analysis; Nonnegative garrote estimator; Numerical analysis; Numerical analysis. Scientific computation; Numerical methods in probability and statistics; Penalized thin plate regression spline; Practical variable selection; Probability and statistics; Sciences and techniques of general use; Shrinkage smoother; Statistics; Penalized thin plate regression spline; Practical variable selection; Generalized additive model; Shrinkage smoother; Nonnegative garrote estimator; Ridge regression; Rank statistic; Shrinkage estimator; Non parametric method; Thin plate; Non parametric estimation; Covariate; Additive model; Penalty method; Parametric method; Statistical test; Cross sectional study; Selection method; Regression spline; Simulation model; Variable selection; Data analysis; Model selection; Empirical method; Statistical estimation; Statistical regression; Selection problem; Statistical computation
• ISSN: 0167-9473; 1872-7352
• DOI: 10.1016/j.csda.2011.02.004

The problem of variable selection within the class of generalized additive models, when there are many covariates to choose from but the number of predictors is still somewhat smaller than the number of observations, is considered. Two very simple but effective shrinkage methods and an extension of the nonnegative garrote estimator are introduced. The proposals avoid having to use nonparametric testing methods for which there is no general reliable distributional theory. Moreover, component selection is carried out in one single step as opposed to many selection procedures which involve an exhaustive search of all possible models. The empirical performance of the proposed methods is compared to that of some available techniques via an extensive simulation study. The results show under which conditions one method can be preferred over another, hence providing applied researchers with some practical guidelines. The procedures are also illustrated analysing data on plasma beta-carotene levels from a cross-sectional study conducted in the United States.