T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-World

TL;DR

This paper introduces T-QPM, a novel framework that enhances vision-language models' ability to detect out-of-distribution data and handle temporal distribution shifts in open-world scenarios through dynamic, cross-modal reasoning and regularization.

Contribution

The paper proposes T-QPM, extending dual-pattern matching with temporal quadruple-patterns and adaptive fusion weights to improve OOD detection and robustness in evolving environments.

Findings

Outperforms static baselines on temporally partitioned benchmarks.

Effectively detects OOD data in non-stationary environments.

Enhances robustness against covariate shifts in vision-language models.

Abstract

Out-of-distribution (OOD) detection remains a critical challenge in open-world learning, where models must adapt to evolving data distributions. While recent vision-language models (VLMS) like CLIP enable multimodal OOD detection through Dual-Pattern Matching (DPM), existing methods typically suffer from two major shortcomings: (1) They rely on fixed fusion rules and assume static environments, failing under temporal drift; and (2) they lack robustness against covariate shifted inputs. In this paper, we propose a novel two-step framework to enhance OOD detection and covariate distribution shift robustness in dynamic settings. We extend the dual-pattern regime into Temporal Quadruple-Pattern Matching (T-QPM). First, by pairing OOD images with text descriptions, we introduce cross-modal consistency patterns between ID and OOD signals, refining the decision boundary through joint image-text reasoning. Second, we address temporal distribution shifts by learning lightweight fusion weights to optimally combine semantic matching and visual typicality. To ensure stability, we enforce explicit regularization based on Average Thresholded Confidence (ATC), preventing performance degradation as distributions evolve. Experiments on temporally partitioned benchmarks demonstrate that our approach significantly outperforms static baselines, offering a robust, temporally-consistent framework for multimodal OOD detection in non-stationary environments.