KVNN: Learnable Multi-Kernel Volterra Neural Networks

TL;DR

This paper introduces KVNN, a learnable multi-kernel Volterra neural network that enhances higher-order feature learning with efficient, order-adaptive kernels, achieving competitive performance with reduced complexity.

Contribution

The paper proposes a kernelized Volterra neural network with multi-kernel, order-adaptive layers that replace standard convolutions, improving efficiency and expressivity in deep learning models.

Findings

kVNN reduces model parameters and GFLOPs compared to standard models.

kVNN achieves competitive or improved performance on video recognition and image denoising.

The approach maintains effectiveness even without large-scale pretraining.

Abstract

Higher-order learning is fundamentally rooted in exploiting compositional features. It clearly hinges on enriching the representation by more elaborate interactions of the data which, in turn, tends to increase the model complexity of conventional large-scale deep learning models. In this paper, a kernelized Volterra Neural Network (kVNN) is proposed. The key to the achieved efficiency lies in using a learnable multi-kernel representation, where different interaction orders are modeled by distinct polynomial-kernel components with compact, learnable centers, yielding an order-adaptive parameterization. Features are learned by the composition of layers, each of which consists of parallel branches of different polynomial orders, enabling kVNN filters to directly replace standard convolutional kernels within existing architectures. The theoretical results are substantiated by experiments on two representative tasks: video action recognition and image denoising. The results demonstrate favorable performance-efficiency trade-offs: kVNN consistently yields reduced model (parameters) and computational (GFLOPs) complexity with competitive and often improved performance. These results are maintained even when trained from scratch without large-scale pretraining. In summary, we substantiate that structured kernelized higher-order layers offer a practical path to balancing expressivity and computational cost in modern deep networks.