Weight Predictor Network with Feature Selection for Small Sample Tabular Biomedical Data

TL;DR

This paper introduces WPFS, a neural network approach with feature selection designed for high-dimensional, small-sample biomedical tabular data, reducing overfitting and improving performance.

Contribution

The paper presents WPFS, a novel neural network with auxiliary networks for feature selection, specifically tailored for small, high-dimensional biomedical datasets, outperforming existing methods.

Findings

WPFS outperforms standard and recent methods on nine biomedical datasets.

Feature selection improves model performance and interpretability.

WPFS reduces overfitting in small-sample, high-dimensional data.

Abstract

Tabular biomedical data is often high-dimensional but with a very small number of samples. Although recent work showed that well-regularised simple neural networks could outperform more sophisticated architectures on tabular data, they are still prone to overfitting on tiny datasets with many potentially irrelevant features. To combat these issues, we propose Weight Predictor Network with Feature Selection (WPFS) for learning neural networks from high-dimensional and small sample data by reducing the number of learnable parameters and simultaneously performing feature selection. In addition to the classification network, WPFS uses two small auxiliary networks that together output the weights of the first layer of the classification model. We evaluate on nine real-world biomedical datasets and demonstrate that WPFS outperforms other standard as well as more recent methods typically applied to tabular data. Furthermore, we investigate the proposed feature selection mechanism and show that it improves performance while providing useful insights into the learning task.