SPlit: An Optimal Method for Data Splitting

TL;DR

SPlit is an optimal data splitting method based on Support Points that improves test performance by systematically selecting representative training and testing subsets for both regression and classification tasks.

Contribution

The paper introduces SPlit, a novel data splitting technique leveraging Support Points, adapted for categorical variables, and demonstrates its effectiveness over random splitting.

Findings

Substantial improvement in worst-case testing performance

Effective for both regression and classification

Outperforms random splitting in real datasets

Abstract

In this article we propose an optimal method referred to as SPlit for splitting a dataset into training and testing sets. SPlit is based on the method of Support Points (SP), which was initially developed for finding the optimal representative points of a continuous distribution. We adapt SP for subsampling from a dataset using a sequential nearest neighbor algorithm. We also extend SP to deal with categorical variables so that SPlit can be applied to both regression and classification problems. The implementation of SPlit on real datasets shows substantial improvement in the worst-case testing performance for several modeling methods compared to the commonly used random splitting procedure.