Improving Out-of-Distribution Data Handling and Corruption Resistance via Modern Hopfield Networks

TL;DR

This paper demonstrates that integrating Modern Hopfield Networks into computer vision models significantly enhances their robustness against out-of-distribution data and corruptions, achieving state-of-the-art results without test-time adaptation.

Contribution

It introduces a novel method of incorporating MHN into models during testing to improve robustness against corruptions, without needing test-time adaptation or augmentation.

Findings

13.84% increase in average corruption accuracy

57.49% reduction in mean Corruption Error (mCE)

60.61% reduction in relative mCE

Abstract

This study explores the potential of Modern Hopfield Networks (MHN) in improving the ability of computer vision models to handle out-of-distribution data. While current computer vision models can generalize to unseen samples from the same distribution, they are susceptible to minor perturbations such as blurring, which limits their effectiveness in real-world applications. We suggest integrating MHN into the baseline models to enhance their robustness. This integration can be implemented during the test time for any model and combined with any adversarial defense method. Our research shows that the proposed integration consistently improves model performance on the MNIST-C dataset, achieving a state-of-the-art increase of 13.84% in average corruption accuracy, a 57.49% decrease in mean Corruption Error (mCE), and a 60.61% decrease in relative mCE compared to the baseline model. Additionally, we investigate the capability of MHN to converge to the original non-corrupted data. Notably, our method does not require test-time adaptation or augmentation with corruptions, underscoring its practical viability for real-world deployment. (Source code publicly available at: https://github.com/salehsargolzaee/Hopfield-integrated-test)