Frequentist forecasting in regime-switching models with extended Hamilton filter

TL;DR

This paper introduces a novel frequentist filtering method for regime-switching state-space models that incorporates both intra- and inter-individual predictors, enabling real-time forecasting of latent states in longitudinal data.

Contribution

It develops the first frequentist filter for RSSS models that depend on individual characteristics, extending existing Bayesian approaches to a new class of models.

Findings

Effective in forecasting student dropout-related emotions and behaviors

Accurate parameter recovery demonstrated in simulations

Improves real-time inference in regime-switching models

Abstract

Psychological change processes, such as university student dropout in math, often exhibit discrete latent state transitions and can be studied using regime-switching models with intensive longitudinal data (ILD). Recently, regime-switching state-space (RSSS) models have been extended to allow for latent variables and their autoregressive effects. Despite this progress, estimation methods for handling both intra-individual changes and inter-individual differences as predictors of regime-switches need further exploration. Specifically, there's a need for frequentist estimation methods in dynamic latent variable frameworks that allow real-time inferences and forecasts of latent or observed variables during ongoing data collection. Building on Chow and Zhang's (2013) extended Kim filter, we introduce a first frequentist filter for RSSS models which allows hidden Markov(-switching) models to depend on both latent within- and between-individual characteristics. As a counterpart of Kelava et al.'s (2022) Bayesian forecasting filter for nonlinear dynamic latent class structural equation models (NDLC-SEM), our proposed method is the first frequentist approach within this general class of models. In an empirical study, the filter is applied to forecast emotions and behavior related to student dropout in math. Parameter recovery and prediction of regime and dynamic latent variables are evaluated through simulation study.