Principled Input-Output-Conditioned Post-Hoc Uncertainty Estimation for Regression Networks

TL;DR

This paper introduces a theoretically grounded post-hoc method for uncertainty estimation in regression networks that leverages auxiliary models and diverse data to improve out-of-distribution detection and uncertainty calibration.

Contribution

We propose a novel framework for post-hoc uncertainty estimation in regression that does not require model retraining or gradients, using auxiliary models and diverse data to enhance OOD detection.

Findings

Using diverse auxiliary data improves OOD detection.

The method maintains proper input-dependent uncertainty estimation.

The approach is validated on toy, UCI, and depth regression benchmarks.

Abstract

Uncertainty quantification is critical in safety-sensitive applications but is often omitted from off-the-shelf neural networks due to adverse effects on predictive performance. Retrofitting uncertainty estimates post-hoc typically requires access to model parameters or gradients, limiting feasibility in practice. We propose a theoretically grounded framework for post-hoc uncertainty estimation in regression tasks by fitting an auxiliary model to both original inputs and frozen model outputs. Drawing from principles of maximum likelihood estimation and sequential parameter fitting, we formalize an exact post-hoc optimization objective that recovers the canonical MLE of Gaussian parameters, without requiring sampling or approximation at inference. While prior work has used model outputs to estimate uncertainty, we explicitly characterize the conditions under which this is valid and demonstrate the extent to which structured outputs can support quasi-epistemic inference. We find that using diverse auxiliary data, such as augmented subsets of the original training data, significantly enhances OOD detection and metric performance. Our hypothesis that frozen model outputs contain generalizable latent information about model error and predictive uncertainty is tested and confirmed. Finally, we ensure that our method maintains proper estimation of input-dependent uncertainty without relying exclusively on base model forecasts. These findings are demonstrated in toy problems and adapted to both UCI and depth regression benchmarks. Code: https://github.com/biggzlar/IO-CUE.