CNN as Guided Multi-layer RECOS Transform

TL;DR

This paper introduces the RECOS transform as a fundamental component of CNNs, providing a new interpretation of CNNs as guided multi-layer RECOS transforms, and discusses their training and testing simplifications.

Contribution

It presents the RECOS transform as a new building block for CNNs and offers a comprehensive explanation of CNN operation and training.

Findings

RECOS transform as a fundamental CNN component

CNN operates as guided multi-layer RECOS transform

Trained CNNs can be simplified to one-bit representations

Abstract

There is a resurging interest in developing a neural-network-based solution to the supervised machine learning problem. The convolutional neural network (CNN) will be studied in this note. To begin with, we introduce a RECOS transform as a basic building block of CNNs. The "RECOS" is an acronym for "REctified-COrrelations on a Sphere". It consists of two main concepts: 1) data clustering on a sphere and 2) rectification. Afterwards, we interpret a CNN as a network that implements the guided multi-layer RECOS transform with three highlights. First, we compare the traditional single-layer and modern multi-layer signal analysis approaches, point out key ingredients that enable the multi-layer approach, and provide a full explanation to the operating principle of CNNs. Second, we discuss how guidance is provided by labels through backpropagation (BP) in the training. Third, we show that a trained network can be greatly simplified in the testing stage demanding only one-bit representation for both filter weights and inputs.