Regularized Nonlinear Regression for Simultaneously Selecting and Estimating Key Model Parameters

TL;DR

This paper introduces a regularized nonlinear regression method that simultaneously selects and estimates key model parameters, improving identifiability and interpretability in system identification with limited data.

Contribution

The paper proposes a novel L1-regularized nonlinear least squares estimator with Levenberg-Marquardt optimization, providing theoretical guarantees and demonstrating effectiveness in biomechanical modeling.

Findings

96.1% reduction in parameter variance in simulations

Maintains over 82.5% variance accounted for in experiments

54 times faster optimization compared to standard methods

Abstract

In system identification, estimating parameters of a model using limited observations results in poor identifiability. To cope with this issue, we propose a new method to simultaneously select and estimate sensitive parameters as key model parameters and fix the remaining parameters to a set of typical values. Our method is formulated as a nonlinear least squares estimator with L1-regularization on the deviation of parameters from a set of typical values. First, we provide consistency and oracle properties of the proposed estimator as a theoretical foundation. Second, we provide a novel approach based on Levenberg-Marquardt optimization to numerically find the solution to the formulated problem. Third, to show the effectiveness, we present an application identifying a biomechanical parametric model of a head position tracking task for 10 human subjects from limited data. In a simulation study, the variances of estimated parameters are decreased by 96.1% as compared to that of the estimated parameters without L1-regularization. In an experimental study, our method improves the model interpretation by reducing the number of parameters to be estimated while maintaining variance accounted for (VAF) at above 82.5%. Moreover, the variances of estimated parameters are reduced by 71.1% as compared to that of the estimated parameters without L1-regularization. Our method is 54 times faster than the standard simplex-based optimization to solve the regularized nonlinear regression.