A Matrix Ensemble Kalman Filter-based Multi-arm Neural Network to Adequately Approximate Deep Neural Networks

TL;DR

This paper introduces MEnKF-ANN, a matrix ensemble Kalman filter-based multi-arm neural network that effectively approximates deep neural networks, including LSTMs, especially with small sample sizes, and provides uncertainty estimates.

Contribution

It presents a novel multi-arm Kalman filter approach for neural network approximation that handles unequal feature set sizes and offers uncertainty quantification.

Findings

Successfully approximates LSTM networks for microbiome classification

Performs well with small training samples

Provides reliable uncertainty estimates in predictions

Abstract

Deep Learners (DLs) are the state-of-art predictive mechanism with applications in many fields requiring complex high dimensional data processing. Although conventional DLs get trained via gradient descent with back-propagation, Kalman Filter (KF)-based techniques that do not need gradient computation have been developed to approximate DLs. We propose a multi-arm extension of a KF-based DL approximator that can mimic DL when the sample size is too small to train a multi-arm DL. The proposed Matrix Ensemble Kalman Filter-based multi-arm ANN (MEnKF-ANN) also performs explicit model stacking that becomes relevant when the training sample has an unequal-size feature set. Our proposed technique can approximate Long Short-term Memory (LSTM) Networks and attach uncertainty to the predictions obtained from these LSTMs with desirable coverage. We demonstrate how MEnKF-ANN can "adequately" approximate an LSTM network trained to classify what carbohydrate substrates are digested and utilized by a microbiome sample whose genomic sequences consist of polysaccharide utilization loci (PULs) and their encoded genes.