Penalized estimation for non-identifiable models

TL;DR

This paper develops asymptotic theory for penalized estimators in singular models where parameters may be non-identifiable or on the boundary, ensuring selection consistency and oracle properties.

Contribution

It introduces a framework for penalized estimation in non-identifiable models, demonstrating convergence and model selection consistency even with complex parametric structures.

Findings

Penalized estimators achieve convergence to the most parsimonious true parameter values.

Selection consistency is validated for singular models with boundary parameters.

Oracle properties are established for non-convex penalties like Bridge and adaptive Lasso.

Abstract

We derive asymptotic properties of penalized estimators for singular models for which identifiability may break and the true parameter values can lie on the boundary of the parameter space. Selection consistency of the estimators is also validated. The problem that the true values lie on the boundary is dealt with by our previous results that are applicable to singular models, besides, penalized estimation and non-ergodic statistics. In order to overcome non-identifiability, we consider a suitable penalty such as the non-convex Bridge and the adaptive Lasso that stabilizes the asymptotic behavior of the estimator and shrinks inactive parameters. Then the estimator converges to one of the most parsimonious values among all the true values. In particular, the oracle property can also be obtained even if parametric structure of the singular model is so complex that likelihood ratio tests for model selection are labor intensive to perform. Among many potential applications, the examples handled in the paper are: (i) the superposition of parametric proportional hazard models and (ii) a counting process having intensity with multicollinear covariates.