Similarity-Distance-Magnitude Universal Verification

TL;DR

This paper introduces the SDM activation function that enhances neural network robustness by providing reliable uncertainty estimates, improving interpretability, and maintaining performance under distribution shifts, with applications in selective classification and model verification.

Contribution

The paper proposes the SDM activation function, integrating similarity, distance, and magnitude awareness, to improve uncertainty estimation and robustness in neural networks, along with methods for interpretability and out-of-distribution detection.

Findings

SDM provides robust uncertainty estimates under distribution shifts.

The approach improves interpretability through class-conditional calibration.

Open-source implementation is provided for practical use.

Abstract

We address the neural network robustness problem by adding Similarity (i.e., correctly predicted depth-matches into training)-awareness and Distance-to-training-distribution-awareness to the existing output Magnitude (i.e., decision-boundary)-awareness of the softmax function. The resulting SDM activation function provides strong signals of the relative epistemic (reducible) predictive uncertainty. We use this novel behavior to further address the complementary HCI problem of mapping the output to human-interpretable summary statistics over relevant partitions of a held-out calibration set. Estimates of prediction-conditional uncertainty are obtained via a parsimonious learned transform over the class-conditional empirical CDFs of the output of a final-layer SDM activation function. For decision-making and as an intrinsic model check, estimates of class-conditional accuracy are obtained by further partitioning the high-probability regions of this calibrated output into class-conditional, region-specific CDFs. The uncertainty estimates from SDM calibration are remarkably robust to test-time distribution shifts and out-of-distribution inputs; incorporate awareness of the effective sample size; provide estimates of uncertainty from the learning and data splitting processes; and are well-suited for selective classification and conditional branching for additional test-time compute based on the predictive uncertainty, as for selective LLM generation, routing, and composition over multiple models and retrieval. Finally, we construct SDM networks, LLMs with uncertainty-aware verification and interpretability-by-exemplar as intrinsic properties. We provide open-source software implementing these results.