Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks

TL;DR

This paper introduces SAL-GP, a structured layerwise Gaussian Process calibration method that aligns with neural network hierarchies to improve interpretability and reliability of predictive uncertainty estimation.

Contribution

It proposes a novel multi-layer GP framework with structured kernels that mirror neural network layers for better calibration and interpretability.

Findings

Enhanced calibration accuracy over traditional methods

Improved interpretability aligned with network structure

Effective uncertainty propagation across layers

Abstract

Calibrating the confidence of neural network classifiers is essential for quantifying the reliability of their predictions during inference. However, conventional Gaussian Process (GP) calibration methods often fail to capture the internal hierarchical structure of deep neural networks, limiting both interpretability and effectiveness for assessing predictive reliability. We propose a Semantic-Aware Layer-wise Gaussian Process (SAL-GP) framework that mirrors the layered architecture of the target neural network. Instead of applying a single global GP correction, SAL-GP employs a multi-layer GP model, where each layer's feature representation is mapped to a local calibration correction. These layerwise GPs are coupled through a structured multi-layer kernel, enabling joint marginalization across all layers. This design allows SAL-GP to capture both local semantic dependencies and global calibration coherence, while consistently propagating predictive uncertainty through the network. The resulting framework enhances interpretability aligned with the network architecture and enables principled evaluation of confidence consistency and uncertainty quantification in deep models.