Contextual fusion enhances robustness to image blurring

TL;DR

This paper introduces a fusion model that combines features from different CNNs to improve robustness against image blurring and perturbations, inspired by mammalian brain integration, showing enhanced performance on MS COCO.

Contribution

The paper presents a novel multi-modal fusion approach that enhances robustness to image perturbations, outperforming traditional single-modality models.

Findings

Fusion model improves robustness to image perturbations.

Context variability correlates with robustness gains.

Fusion approach complements existing defense methods.

Abstract

Mammalian brains handle complex reasoning by integrating information across brain regions specialized for particular sensory modalities. This enables improved robustness and generalization versus deep neural networks, which typically process one modality and are vulnerable to perturbations. While defense methods exist, they do not generalize well across perturbations. We developed a fusion model combining background and foreground features from CNNs trained on Imagenet and Places365. We tested its robustness to human-perceivable perturbations on MS COCO. The fusion model improved robustness, especially for classes with greater context variability. Our proposed solution for integrating multiple modalities provides a new approach to enhance robustness and may be complementary to existing methods.