Reinforced Data Sampling for Model Diversification

TL;DR

This paper introduces Reinforced Data Sampling (RDS), a novel method that uses reinforcement learning to select data subsets promoting model diversity, thereby improving performance in machine learning tasks and competitions.

Contribution

The work proposes a reinforcement learning-based data sampling approach that optimizes model diversification and demonstrates its effectiveness across multiple datasets.

Findings

RDS outperforms traditional sampling methods in experiments.

RDS enhances model diversity and learning potential.

The method is applicable to classification and regression tasks.

Abstract

With the rising number of machine learning competitions, the world has witnessed an exciting race for the best algorithms. However, the involved data selection process may fundamentally suffer from evidence ambiguity and concept drift issues, thereby possibly leading to deleterious effects on the performance of various models. This paper proposes a new Reinforced Data Sampling (RDS) method to learn how to sample data adequately on the search for useful models and insights. We formulate the optimisation problem of model diversification $\delta{-div}$ in data sampling to maximise learning potentials and optimum allocation by injecting model diversity. This work advocates the employment of diverse base learners as value functions such as neural networks, decision trees, or logistic regressions to reinforce the selection process of data subsets with multi-modal belief. We introduce different ensemble reward mechanisms, including soft voting and stochastic choice to approximate optimal sampling policy. The evaluation conducted on four datasets evidently highlights the benefits of using RDS method over traditional sampling approaches. Our experimental results suggest that the trainable sampling for model diversification is useful for competition organisers, researchers, or even starters to pursue full potentials of various machine learning tasks such as classification and regression. The source code is available at https://github.com/probeu/RDS.