Parsimonious Computing: A Minority Training Regime for Effective Prediction in Large Microarray Expression Data Sets

TL;DR

This paper introduces a novel, theoretically grounded method for training shallow neural networks on large microarray datasets, reducing tuning effort and computational costs while improving prediction accuracy.

Contribution

The paper presents a new learning rate computation based on Lipschitz continuity, combined with a novel activation function, for efficient gene expression inference.

Findings

Reduced tuning effort and computational cost

Improved prediction accuracy over existing methods

Effective handling of large microarray datasets

Abstract

Rigorous mathematical investigation of learning rates used in back-propagation in shallow neural networks has become a necessity. This is because experimental evidence needs to be endorsed by a theoretical background. Such theory may be helpful in reducing the volume of experimental effort to accomplish desired results. We leveraged the functional property of Mean Square Error, which is Lipschitz continuous to compute learning rate in shallow neural networks. We claim that our approach reduces tuning efforts, especially when a significant corpus of data has to be handled. We achieve remarkable improvement in saving computational cost while surpassing prediction accuracy reported in literature. The learning rate, proposed here, is the inverse of the Lipschitz constant. The work results in a novel method for carrying out gene expression inference on large microarray data sets with a shallow architecture constrained by limited computing resources. A combination of random sub-sampling of the dataset, an adaptive Lipschitz constant inspired learning rate and a new activation function, A-ReLU helped accomplish the results reported in the paper.