Interleaving cortex-analog mixing improves deep non-negative matrix factorization networks

TL;DR

This paper shows that combining positive and local interactions in deep networks, inspired by the brain's cortex, improves performance over traditional networks.

Contribution

Introducing intermediate modules that mimic cortical column processing to enhance deep NMF networks.

Findings

Intermediate modules combining NMF's positive activities improve benchmark performance.

The approach outperforms conventional deep convolutional networks of similar size.

Positive long-range signaling with local interactions enhances deep network performance.

Abstract

Considering biological constraints in artificial neural networks has led to dramatic improvements in performance. Nevertheless, to date, the positivity of long-range signals in the cortex has not been shown to yield improvements. While Non-negative matrix factorization (NMF) captures biological constraints of positive long-range interactions, deep convolutional neural networks with NMF modules do not match the performance of conventional neural networks (CNNs) of a similar size. This work shows that introducing intermediate modules that combine the NMF's positive activities, analogous to the processing in cortical columns, leads to improved performance on benchmark data that exceeds that of vanilla deep convolutional networks. This demonstrates that including positive long-range signaling together with local interactions of both signs in analogy to cortical hyper-columns has the potential to enhance the performance of deep networks.

Figures