LatentTrack: Sequential Weight Generation via Latent Filtering

TL;DR

LatentTrack introduces a novel online probabilistic prediction method that uses a low-dimensional latent space and hypernetworks for constant-time adaptation, outperforming existing models on climate data.

Contribution

The paper presents LatentTrack, a new sequential neural architecture that performs causal Bayesian filtering in latent space with hypernetworks, enabling efficient online adaptation without gradient updates.

Findings

Lower negative log-likelihood on Jena Climate benchmark

Improved mean squared error over baselines

Competitive calibration performance

Abstract

We introduce LatentTrack (LT), a sequential neural architecture for online probabilistic prediction under nonstationary dynamics. LT performs causal Bayesian filtering in a low-dimensional latent space and uses a lightweight hypernetwork to generate predictive model parameters at each time step, enabling constant-time online adaptation without per-step gradient updates. At each time step, a learned latent model predicts the next latent distribution, which is updated via amortized inference using new observations, yielding a predict--generate--update filtering framework in function space. The formulation supports both structured (Markovian) and unstructured latent dynamics within a unified objective, while Monte Carlo inference over latent trajectories produces calibrated predictive mixtures with fixed per-step cost. Evaluated on long-horizon online regression using the Jena Climate benchmark, LT consistently achieves lower negative log-likelihood and mean squared error than stateful sequential and static uncertainty-aware baselines, with competitive calibration, demonstrating that latent-conditioned function evolution is an effective alternative to traditional latent-state modeling under distribution shift.