Efficient and doubly-robust methods for variable selection and parameter estimation in longitudinal data analysis

• Published in: Computational statistics Vol. 36; no. 2; pp. 781 - 804
• Main Authors: Fu, Liya, Yang, Zhuoran, Cai, Fengjing, Wang, You-Gan
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2021; Springer Nature B.V
• Subjects: Asymptotic methods; Asymptotic properties; Data analysis; Data structures; Datasets; Economic Theory/Quantitative Economics/Mathematical Methods; Feature selection; Massive data points; Mathematics and Statistics; New technology; Original Paper; Outliers (statistics); Parameter estimation; Probability and Statistics in Computer Science; Probability Theory and Stochastic Processes; Robustness (mathematics); Statistical analysis; Statistics; Variables; Outliers; Correlated data; Rank-based method; Variable selection
• DOI: 10.1007/s00180-020-01038-3
• ISSN: 0943-4062

New technologies have produced increasingly complex and massive datasets, such as next generation sequencing and microarray data in biology, dynamic treatment regimes in clinical trials and long-term wide-scale studies in the social sciences. Each study exhibits its unique data structure within individuals, clusters and possibly across time and space. In order to draw valid conclusion from such large dimensional data, we must account for intracluster correlations, varying cluster sizes, and outliers in response and/or covariate domains to achieve valid and efficient inferences. A weighted rank-based method is proposed for selecting variables and estimating parameters simultaneously. The main contribution of the proposed method is four fold: (1) variable selection using adaptive lasso is extended to robust rank regression so that protection against outliers in both response and predictor variables is obtained; (2) within-subject correlations are incorporated so that efficiency of parameter estimation is improved; (3) the computation is convenient via the existing function in statistical software R. (4) the proposed method is proved to have desirable asymptotic properties for fixed number of covariates ( p ). Simulation studies are carried out to evaluate the proposed method for a number of scenarios including the cases when p equals to the number of subjects. The simulation results indicate that the proposed method is efficient and robust. A hormone dataset is analyzed for illustration. By adding additional redundant variables as covariates, the penalty approach and weighting schemes are proven to be effective.