ExGate: Externally Controlled Gating for Feature-based Attention in Artificial Neural Networks

TL;DR

This paper introduces ExGate, a novel externally controlled gating mechanism for feature-based attention in neural networks, improving classification accuracy and error reasoning with minimal added complexity.

Contribution

It presents a new gating method for neural networks that enhances attention mechanisms using external control, leading to better accuracy and error analysis.

Findings

5% accuracy improvement on CIFAR-10

Reduces cross-category misclassifications

Adds minimal parameters to the model

Abstract

Perceptual capabilities of artificial systems have come a long way since the advent of deep learning. These methods have proven to be effective, however they are not as efficient as their biological counterparts. Visual attention is a set of mechanisms that are employed in biological visual systems to ease computational load by only processing pertinent parts of the stimuli. This paper addresses the implementation of top-down, feature-based attention in an artificial neural network by use of externally controlled neuron gating. Our results showed a 5% increase in classification accuracy on the CIFAR-10 dataset versus a non-gated version, while adding very few parameters. Our gated model also produces more reasonable errors in predictions by drastically reducing prediction of classes that belong to a different category to the true class.