Regularized Nonlinear Regression for Simultaneously Selecting and Estimating Key Model Parameters

• Published in: arXiv.org
• Main Authors: Yoon, Kyubaek, You, Hojun, Wei-Ying, Wu, Chae Young Lim, Choi, Jongeun, Boss, Connor, Ramadan, Ahmed, Popovich, John M, Cholewicki, Jacek, Reeves, N Peter, Radcliffe, Clark J
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 02.06.2022
• Subjects: Biomechanics; Computer Science - Learning; Estimation; Mathematical models; Optimization; Parameter estimation; Parameter identification; Parameter sensitivity; Regularization; Statistics - Methodology; System identification
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2104.11426

In system identification, estimating parameters of a model using limited observations results in poor identifiability. To cope with this issue, we propose a new method to simultaneously select and estimate sensitive parameters as key model parameters and fix the remaining parameters to a set of typical values. Our method is formulated as a nonlinear least squares estimator with L1-regularization on the deviation of parameters from a set of typical values. First, we provide consistency and oracle properties of the proposed estimator as a theoretical foundation. Second, we provide a novel approach based on Levenberg-Marquardt optimization to numerically find the solution to the formulated problem. Third, to show the effectiveness, we present an application identifying a biomechanical parametric model of a head position tracking task for 10 human subjects from limited data. In a simulation study, the variances of estimated parameters are decreased by 96.1% as compared to that of the estimated parameters without L1-regularization. In an experimental study, our method improves the model interpretation by reducing the number of parameters to be estimated while maintaining variance accounted for (VAF) at above 82.5%. Moreover, the variances of estimated parameters are reduced by 71.1% as compared to that of the estimated parameters without L1-regularization. Our method is 54 times faster than the standard simplex-based optimization to solve the regularized nonlinear regression.