Soft Rule Ensembles for Statistical Learning

TL;DR

This paper introduces soft rule ensembles for supervised learning, using logistic regression with bias correction to improve predictive accuracy over traditional hard rule ensembles.

Contribution

It presents a novel method combining importance sampling ensembles with soft rules derived via bias-corrected logistic regression, enhancing predictive performance.

Findings

Soft rule ensembles outperform hard rule ensembles in predictive accuracy.

Bias correction with Firth's likelihood addresses perfect separation issues.

Simulation results validate the effectiveness of the proposed approach.

Abstract

In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the perfect separation problem related to the logistic regression, Firth's bias corrected likelihood is used. Various examples and simulation results show that soft rule ensembles can improve predictive performance over hard rule ensembles.