Putting the Iterative Training of Decision Trees to the Test on a Real-World Robotic Task

TL;DR

This paper tests an iterative decision tree training algorithm on a real-world robotic task involving balancing a pendulum, demonstrating that DTs can match DRL performance with fewer parameters in noisy, real-world conditions.

Contribution

First application of an iterative decision tree training algorithm to a real-world robotic task, showing its effectiveness in complex, noisy environments.

Findings

DT performance matches DRL agent in real-world task

Generated DTs are more lightweight with fewer parameters

Algorithm proves robust against noise and delays

Abstract

In previous research, we developed methods to train decision trees (DT) as agents for reinforcement learning tasks, based on deep reinforcement learning (DRL) networks. The samples from which the DTs are built, use the environment's state as features and the corresponding action as label. To solve the nontrivial task of selecting samples, which on one hand reflect the DRL agent's capabilities of choosing the right action but on the other hand also cover enough state space to generalize well, we developed an algorithm to iteratively train DTs. In this short paper, we apply this algorithm to a real-world implementation of a robotic task for the first time. Real-world tasks pose additional challenges compared to simulations, such as noise and delays. The task consists of a physical pendulum attached to a cart, which moves on a linear track. By movements to the left and to the right, the pendulum is to be swung in the upright position and balanced in the unstable equilibrium. Our results demonstrate the applicability of the algorithm to real-world tasks by generating a DT whose performance matches the performance of the DRL agent, while consisting of fewer parameters. This research could be a starting point for distilling DTs from DRL agents to obtain transparent, lightweight models for real-world reinforcement learning tasks.