An Equivalence of Fully Connected Layer and Convolutional Layer

TL;DR

This paper shows that convolutional layers can be mathematically transformed into fully connected layers via matrix multiplication, clarifying their underlying equivalence for neural network understanding.

Contribution

It provides a clear demonstration of the equivalence between convolutional and fully connected layers in the linear case, aiding beginners' comprehension.

Findings

Convolutional operations can be converted to matrix multiplication.

The equivalence holds in the linear case and can be extended to non-linear cases.

Helps beginners understand neural network layer operations.

Abstract

This article demonstrates that convolutional operation can be converted to matrix multiplication, which has the same calculation way with fully connected layer. The article is helpful for the beginners of the neural network to understand how fully connected layer and the convolutional layer work in the backend. To be concise and to make the article more readable, we only consider the linear case. It can be extended to the non-linear case easily through plugging in a non-linear encapsulation to the values like this $\sigma(x)$ denoted as $x^{\prime}$.