General OOD Detection via Model-aware and Subspace-aware Variable Priority

TL;DR

This paper proposes a novel, model-aware and subspace-aware framework for out-of-distribution detection applicable to various outcome types, improving detection accuracy without relying on global density estimates.

Contribution

It introduces a new OOD detection method that embeds variable prioritization into the detection process, applicable to regression and survival analysis, and demonstrates its effectiveness on synthetic, real data, and clinical studies.

Findings

Consistent improvements over existing methods on synthetic and real benchmarks.

Effective detection of functional shifts in diverse outcome models.

Application to clinical data reveals meaningful distribution shifts.

Abstract

Out-of-distribution (OOD) detection is essential for determining when a supervised model encounters inputs that differ meaningfully from its training distribution. While widely studied in classification, OOD detection for regression and survival analysis remains limited due to the absence of discrete labels and the challenge of quantifying predictive uncertainty. We introduce a framework for OOD detection that is simultaneously model aware and subspace aware, and that embeds variable prioritization directly into the detection step. The method uses the fitted predictor to construct localized neighborhoods around each test case that emphasize the features driving the model's learned relationship and downweight directions that are less relevant to prediction. It produces OOD scores without relying on global distance metrics or estimating the full feature density. The framework is applicable across outcome types, and in our implementation we use random forests, where the rule structure yields transparent neighborhoods and effective scoring. Experiments on synthetic and real data benchmarks designed to isolate functional shifts show consistent improvements over existing methods. We further demonstrate the approach in an esophageal cancer survival study, where distribution shifts related to lymphadenectomy identify patterns relevant to surgical guidelines.