JSRT: James-Stein Regression Tree

TL;DR

The paper introduces JSRT, a novel regression tree that leverages global mean information via James-Stein estimator to improve prediction accuracy, validated through extensive experiments on benchmark datasets.

Contribution

We propose the James-Stein Regression Tree (JSRT), which incorporates global information from multiple nodes during construction and prediction, enhancing traditional regression tree performance.

Findings

JSRT outperforms existing regression tree methods in accuracy.

The method effectively utilizes global mean information for better predictions.

Experimental results demonstrate improved generalization and efficiency.

Abstract

Regression tree (RT) has been widely used in machine learning and data mining community. Given a target data for prediction, a regression tree is first constructed based on a training dataset before making prediction for each leaf node. In practice, the performance of RT relies heavily on the local mean of samples from an individual node during the tree construction/prediction stage, while neglecting the global information from different nodes, which also plays an important role. To address this issue, we propose a novel regression tree, named James-Stein Regression Tree (JSRT) by considering global information from different nodes. Specifically, we incorporate the global mean information based on James-Stein estimator from different nodes during the construction/predicton stage. Besides, we analyze the generalization error of our method under the mean square error (MSE) metric. Extensive experiments on public benchmark datasets verify the effectiveness and efficiency of our method, and demonstrate the superiority of our method over other RT prediction methods.