Exploring Loss Design Techniques For Decision Tree Robustness To Label Noise

TL;DR

This paper investigates how loss design techniques from deep learning can be adapted to enhance decision tree robustness against label noise, revealing that standard methods like loss correction are ineffective and suggesting new directions.

Contribution

The study evaluates existing loss correction methods for decision trees under label noise and highlights the need for novel approaches beyond standard techniques.

Findings

Loss correction and symmetric losses are ineffective for decision trees with noisy labels.

Standard deep learning loss techniques do not translate well to decision tree robustness.

New research directions are necessary to improve decision tree resilience to label noise.

Abstract

In the real world, data is often noisy, affecting not only the quality of features but also the accuracy of labels. Current research on mitigating label errors stems primarily from advances in deep learning, and a gap exists in exploring interpretable models, particularly those rooted in decision trees. In this study, we investigate whether ideas from deep learning loss design can be applied to improve the robustness of decision trees. In particular, we show that loss correction and symmetric losses, both standard approaches, are not effective. We argue that other directions need to be explored to improve the robustness of decision trees to label noise.