Two-Stage Regularization of Pseudo-Likelihood Estimators with Application to Time Series

TL;DR

This paper introduces a two-stage regularization method for pseudo-likelihood estimators, improving approximation to the true likelihood, with applications to time series and OLS estimation under exogeneity.

Contribution

It proposes a novel two-stage regularization approach that better approximates the likelihood for pseudo-likelihood estimators, enhancing estimation accuracy.

Findings

Improved regularization method for pseudo-likelihood estimators.

Application to time series OLS under exogeneity.

Theoretical justification for the two-stage approach.

Abstract

Estimators derived from score functions that are not the likelihood are in wide use in practical and modern applications. Their regularization is often carried by pseudo-posterior estimation, equivalently by adding penalty to the score function. We argue that this approach is suboptimal, and propose a two-staged alternative involving estimation of a new score function which better approximates the true likelihood for the purpose of regularization. Our approach typically identifies with maximum a-posteriori estimation if the original score function is in fact the likelihood. We apply our theory to fitting ordinary least squares (OLS) under contemporaneous exogeneity, a setting appearing often in time series and in which OLS is the estimator of choice by practitioners.