Bio-inspired Min-Nets Improve the Performance and Robustness of Deep Networks

TL;DR

This paper introduces Min-Nets, inspired by cortical cells, which improve deep network performance and robustness by incorporating minimum operations, acting as an effective AND operation on filters, especially beneficial for natural image statistics.

Contribution

The paper proposes Min-Nets with Min-units integrated into ResNet and DenseNet, demonstrating enhanced accuracy and robustness against JPEG artifacts.

Findings

Min-Nets outperform standard networks on Cifar-10

Min-Nets show increased robustness to JPEG compression

Minimum operations act as effective AND functions on filters

Abstract

Min-Nets are inspired by end-stopped cortical cells with units that output the minimum of two learned filters. We insert such Min-units into state-of-the-art deep networks, such as the popular ResNet and DenseNet, and show that the resulting Min-Nets perform better on the Cifar-10 benchmark. Moreover, we show that Min-Nets are more robust against JPEG compression artifacts. We argue that the minimum operation is the simplest way of implementing an AND operation on pairs of filters and that such AND operations introduce a bias that is appropriate given the statistics of natural images.