Volterra Neural Networks (VNNs)

TL;DR

This paper introduces Volterra Neural Networks (VNNs), a novel architecture inspired by Volterra filters that reduces complexity and parameters, demonstrating superior performance in action recognition tasks compared to traditional CNNs.

Contribution

The paper proposes a new VNN architecture with cascaded Volterra filtering, enabling efficient parameter reduction and improved action recognition performance.

Findings

VNN outperforms state-of-the-art CNNs on UCF-101 and HMDB-51 datasets.

The architecture effectively fuses spatial and temporal information in videos.

VNN maintains simplicity and tractability while achieving high accuracy.

Abstract

The importance of inference in Machine Learning (ML) has led to an explosive number of different proposals in ML, and particularly in Deep Learning. In an attempt to reduce the complexity of Convolutional Neural Networks, we propose a Volterra filter-inspired Network architecture. This architecture introduces controlled non-linearities in the form of interactions between the delayed input samples of data. We propose a cascaded implementation of Volterra Filtering so as to significantly reduce the number of parameters required to carry out the same classification task as that of a conventional Neural Network. We demonstrate an efficient parallel implementation of this Volterra Neural Network (VNN), along with its remarkable performance while retaining a relatively simpler and potentially more tractable structure. Furthermore, we show a rather sophisticated adaptation of this network to nonlinearly fuse the RGB (spatial) information and the Optical Flow (temporal) information of a video sequence for action recognition. The proposed approach is evaluated on UCF-101 and HMDB-51 datasets for action recognition, and is shown to outperform state of the art CNN approaches.