Bilinear Models for Machine Learning

TL;DR

This paper introduces bilinear models for machine learning that better exploit data structure, such as images, leading to fewer parameters and improved efficiency in tasks like MNIST classification.

Contribution

It proposes bilinear operations as a replacement for linear ones in ML models, enhancing data structure utilization and reducing parameter count.

Findings

Bilinear models outperform linear models on MNIST classification.

Fewer parameters are needed for similar or better performance.

Bilinear operations better capture spatial relationships in images.

Abstract

In this work we define and analyze the bilinear models which replace the conventional linear operation used in many building blocks of machine learning (ML). The main idea is to devise the ML algorithms which are adapted to the objects they treat. In the case of monochromatic images, we show that the bilinear operation exploits better the structure of the image than the conventional linear operation which ignores the spatial relationship between the pixels. This translates into significantly smaller number of parameters required to yield the same performance. We show numerical examples of classification in the MNIST data set.