Robustness of Epinets against Distributional Shifts

TL;DR

This paper evaluates the robustness of epinets, a neural network uncertainty modeling approach, under distributional shifts, showing they outperform large ensembles in robustness metrics at lower computational costs.

Contribution

The study demonstrates that epinets improve robustness to distributional shifts more effectively than large ensembles, with significantly less computational expense.

Findings

Epinets generally improve robustness metrics across ImageNet-A/O/C.

Improvements are more significant than those from very large ensembles.

Epinets are not a complete solution for distributionally-robust deep learning.

Abstract

Recent work introduced the epinet as a new approach to uncertainty modeling in deep learning. An epinet is a small neural network added to traditional neural networks, which, together, can produce predictive distributions. In particular, using an epinet can greatly improve the quality of joint predictions across multiple inputs, a measure of how well a neural network knows what it does not know. In this paper, we examine whether epinets can offer similar advantages under distributional shifts. We find that, across ImageNet-A/O/C, epinets generally improve robustness metrics. Moreover, these improvements are more significant than those afforded by even very large ensembles at orders of magnitude lower computational costs. However, these improvements are relatively small compared to the outstanding issues in distributionally-robust deep learning. Epinets may be a useful tool in the toolbox, but they are far from the complete solution.