Use of Deterministic Transforms to Design Weight Matrices of a Neural Network

TL;DR

This paper investigates replacing random matrices with deterministic transforms like DCT, Hadamard, Hartley, and wavelet in SSFN neural networks, reducing computational complexity and enabling unsupervised transform selection for improved object classification.

Contribution

It introduces a novel approach using deterministic transforms in SSFN, with methods for unsupervised transform selection based on statistical features, enhancing efficiency and adaptability.

Findings

Deterministic transforms reduce computational complexity.

Unsupervised transform selection improves classification performance.

The approach is effective on benchmark datasets.

Abstract

Self size-estimating feedforward network (SSFN) is a feedforward multilayer network. For the existing SSFN, a part of each weight matrix is trained using a layer-wise convex optimization approach (a supervised training), while the other part is chosen as a random matrix instance (an unsupervised training). In this article, the use of deterministic transforms instead of random matrix instances for the SSFN weight matrices is explored. The use of deterministic transforms provides a reduction in computational complexity. The use of several deterministic transforms is investigated, such as discrete cosine transform, Hadamard transform, Hartley transform, and wavelet transforms. The choice of a deterministic transform among a set of transforms is made in an unsupervised manner. To this end, two methods based on features' statistical parameters are developed. The proposed methods help to design a neural net where deterministic transforms can vary across its layers' weight matrices. The effectiveness of the proposed approach vis-a-vis the SSFN is illustrated for object classification tasks using several benchmark datasets.