Stochastic Approximation Algorithm for Estimating Mixing Distribution for Dependent Observations

TL;DR

This paper extends the stochastic approximation algorithm for estimating mixing densities to dependent observations, proving its consistency under certain weak dependence conditions, thus broadening its applicability in statistical mixture models.

Contribution

It demonstrates that Newton's recursive algorithm remains consistent for dependent data, under specific decay conditions on the dependence structure.

Findings

Algorithm remains consistent with weak dependence among observations.

Consistency proven under decay condition on dependence similar to mutual information.

Extends applicability of stochastic approximation methods to dependent data scenarios.

Abstract

Estimating the mixing density of a mixture distribution remains an interesting problem in statistics literature. Using a stochastic approximation method, Newton and Zhang (1999) introduced a fast recursive algorithm for estimating the mixing density of a mixture. Under suitably chosen weights the stochastic approximation estimator converges to the true solution. In Tokdar et. al. (2009) the consistency of this recursive estimation method was established. However, the proof of consistency of the resulting estimator used independence among observations as an assumption. Here, we extend the investigation of performance of Newton's algorithm to several dependent scenarios. We prove that the original algorithm under certain conditions remains consistent even when the observations are arising from a weakly dependent stationary process with the target mixture as the marginal density. We show consistency under a decay condition on the dependence among observations when the dependence is characterized by a quantity similar to mutual information between the observations.