Lattice: A Confidence-Gated Hybrid System for Uncertainty-Aware Sequential Prediction with Behavioral Archetypes

TL;DR

Lattice is a hybrid system that uses confidence gating to activate behavioral archetypes for improved sequential prediction, effectively managing uncertainty across diverse domains like recommendation, scientific data, and finance.

Contribution

The paper introduces Lattice, a novel confidence-gated hybrid system that selectively activates learned behavioral structures to enhance prediction accuracy and uncertainty management.

Findings

Significant improvement in recommendation accuracy on MovieLens.

Effective detection of distribution shifts in scientific and financial data.

Graceful deferral when structure is redundant or uncertain.

Abstract

We introduce Lattice, a hybrid sequential prediction system that conditionally activates learned behavioral structure using binary confidence gating. The system clusters behavior windows into behavioral archetypes and uses binary confidence gating to activate archetype-based scoring only when confidence exceeds a threshold, falling back to baseline predictions when uncertain. We validate Lattice on recommendation systems (MovieLens), scientific time-series (LIGO), and financial markets, using LSTM and transformer backbones. On MovieLens with LSTM, Lattice achieves +31.9% improvement over LSTM baseline in HR@10 (p < 3.29 x 10^-25, 30 seeds), outperforming transformer baselines by 109.4% over SASRec and 218.6% over BERT4Rec. On LIGO and financial data, the system correctly refuses archetype activation when distribution shift occurs - a successful outcome demonstrating confidence gating prevents false activation. On transformer backbones, Lattice provides 0.0% improvement (neutral, no degradation), gracefully deferring when structure is already present. This bidirectional validation - activating when patterns apply, refusing when they don't, and deferring when redundant - supports confidence gating as a promising architectural principle for managing epistemic uncertainty in safety-critical applications.