Probabilistic Value Selection for Space Efficient Model

TL;DR

This paper introduces a novel preprocessing technique called Value Selection (VS) that reduces model size while maintaining accuracy by selectively eliminating feature values, supported by probabilistic methods and extensive benchmark experiments.

Contribution

It proposes a new value selection method based on probabilistic information theory, offering an alternative to feature and instance selection for model size reduction.

Findings

Value selection balances accuracy and model size effectively.

Probabilistic methods PVS and P+VS outperform traditional preprocessing techniques.

Extensive experiments validate the approach on various benchmark datasets.

Abstract

An alternative to current mainstream preprocessing methods is proposed: Value Selection (VS). Unlike the existing methods such as feature selection that removes features and instance selection that eliminates instances, value selection eliminates the values (with respect to each feature) in the dataset with two purposes: reducing the model size and preserving its accuracy. Two probabilistic methods based on information theory's metric are proposed: PVS and P + VS. Extensive experiments on the benchmark datasets with various sizes are elaborated. Those results are compared with the existing preprocessing methods such as feature selection, feature transformation, and instance selection methods. Experiment results show that value selection can achieve the balance between accuracy and model size reduction.