Energy Score-Guided Neural Gaussian Mixture Model for Predictive Uncertainty Quantification

TL;DR

This paper introduces NE-GMM, a neural Gaussian mixture model guided by Energy Score, to improve predictive uncertainty quantification with theoretical guarantees and superior empirical performance.

Contribution

It presents a novel hybrid loss integrating GMM and Energy Score, with proven properties and error bounds, enhancing uncertainty estimation in neural models.

Findings

NE-GMM achieves better calibration and accuracy than existing methods.

Theoretical proof of proper scoring rule properties for the hybrid loss.

Empirical results show superior performance on synthetic and real datasets.

Abstract

Quantifying predictive uncertainty is essential for real world machine learning applications, especially in scenarios requiring reliable and interpretable predictions. Many common parametric approaches rely on neural networks to estimate distribution parameters by optimizing the negative log likelihood. However, these methods often encounter challenges like training instability and mode collapse, leading to poor estimates of the mean and variance of the target output distribution. In this work, we propose the Neural Energy Gaussian Mixture Model (NE-GMM), a novel framework that integrates Gaussian Mixture Model (GMM) with Energy Score (ES) to enhance predictive uncertainty quantification. NE-GMM leverages the flexibility of GMM to capture complex multimodal distributions and leverages the robustness of ES to ensure well calibrated predictions in diverse scenarios. We theoretically prove that the hybrid loss function satisfies the properties of a strictly proper scoring rule, ensuring alignment with the true data distribution, and establish generalization error bounds, demonstrating that the model's empirical performance closely aligns with its expected performance on unseen data. Extensive experiments on both synthetic and real world datasets demonstrate the superiority of NE-GMM in terms of both predictive accuracy and uncertainty quantification.