Consistent Estimation for Partition-wise Regression and Classification Models

TL;DR

This paper introduces a consistent, automatic method for selecting partitions and submodels in partition-wise regression and classification, improving interpretability and model accuracy for complex data.

Contribution

It proposes a novel, statistically consistent procedure for automatically determining the number and boundaries of regions in partition-wise models for regression and classification.

Findings

The partition estimator is proven to be statistically consistent.

The methodology effectively balances model simplicity and fit.

Demonstrated applicability to both regression and classification tasks.

Abstract

Partition-wise models offer a flexible approach for modeling complex and multidimensional data that are capable of producing interpretable results. They are based on partitioning the observed data into regions, each of which is modeled with a simple submodel. The success of this approach highly depends on the quality of the partition, as too large a region could lead to a non-simple submodel, while too small a region could inflate estimation variance. This paper proposes an automatic procedure for choosing the partition (i.e., the number of regions and the boundaries between regions) as well as the submodels for the regions. It is shown that, under the assumption of the existence of a true partition, the proposed partition estimator is statistically consistent. The methodology is demonstrated for both regression and classification problems.