Adaptive Random SubSpace Learning (RSSL) Algorithm for Prediction

TL;DR

This paper introduces an adaptive random subspace learning algorithm (RSSL) that enhances prediction accuracy by combining feature weighting, bootstrap sampling, and flexible base learners, applicable to both regression and classification tasks.

Contribution

The paper proposes a novel, adaptable RSSL framework with multiple weighting schemes and a heuristic subspacing method, improving prediction performance over conventional algorithms.

Findings

Outperforms traditional algorithms in most tested cases.

Effective on datasets with high feature-to-sample ratios.

Robust across simulated and real-world data.

Abstract

We present a novel adaptive random subspace learning algorithm (RSSL) for prediction purpose. This new framework is flexible where it can be adapted with any learning technique. In this paper, we tested the algorithm for regression and classification problems. In addition, we provide a variety of weighting schemes to increase the robustness of the developed algorithm. These different wighting flavors were evaluated on simulated as well as on real-world data sets considering the cases where the ratio between features (attributes) and instances (samples) is large and vice versa. The framework of the new algorithm consists of many stages: first, calculate the weights of all features on the data set using the correlation coefficient and F-statistic statistical measurements. Second, randomly draw n samples with replacement from the data set. Third, perform regular bootstrap sampling (bagging). Fourth, draw without replacement the indices of the chosen variables. The decision was taken based on the heuristic subspacing scheme. Fifth, call base learners and build the model. Sixth, use the model for prediction purpose on test set of the data. The results show the advancement of the adaptive RSSL algorithm in most of the cases compared with the synonym (conventional) machine learning algorithms.