A Statistical Decision-Theoretical Perspective on the Two-Stage Approach to Parameter Estimation

TL;DR

This paper provides a decision-theoretic foundation for the Two-Stage approach in parameter estimation, offering Bayesian and Minimax estimators and demonstrating its application through simulations.

Contribution

It introduces a theoretical justification for the Two-Stage method using statistical decision theory, connecting it with Bayesian and Minimax estimation.

Findings

The decision-theoretic derivation justifies TS as Bayesian and Minimax estimators.

Application to i.i.d. models using data quantiles and linear functions.

Numerical simulations demonstrate the effectiveness of the proposed approach.

Abstract

One of the most important problems in system identification and statistics is how to estimate the unknown parameters of a given model. Optimization methods and specialized procedures, such as Empirical Minimization (EM) can be used in case the likelihood function can be computed. For situations where one can only simulate from a parametric model, but the likelihood is difficult or impossible to evaluate, a technique known as the Two-Stage (TS) Approach can be applied to obtain reliable parametric estimates. Unfortunately, there is currently a lack of theoretical justification for TS. In this paper, we propose a statistical decision-theoretical derivation of TS, which leads to Bayesian and Minimax estimators. We also show how to apply the TS approach on models for independent and identically distributed samples, by computing quantiles of the data as a first step, and using a linear function as the second stage. The proposed method is illustrated via numerical simulations.