Interpretability by Design for Efficient Multi-Objective Reinforcement Learning

TL;DR

LLE-MORL is a novel multi-objective reinforcement learning method that offers interpretability by design through a locally linear mapping, enabling efficient and high-quality Pareto front generation in continuous control tasks.

Contribution

The paper introduces LLE-MORL, a new approach that combines interpretability with efficiency in multi-objective RL by leveraging local linear maps between parameters and objectives.

Findings

LLE-MORL outperforms state-of-the-art methods in Pareto front quality.

LLE-MORL achieves faster solution generation without extensive retraining.

The approach provides interpretable policy parameters linked to objectives.

Abstract

Multi-objective reinforcement learning (MORL) aims at optimising several, often conflicting goals to improve the flexibility and reliability of RL in practical tasks. This is typically achieved by finding a set of diverse, non-dominated policies that form a Pareto front in the performance space. We introduce LLE-MORL, an approach that achieves interpretability by design by utilising a training scheme based on the local relationship between the parameter space and the performance space. By exploiting a locally linear map between these spaces, our method provides an interpretation of policy parameters in terms of the objectives, and this structured representation enables an efficient search within contiguous solution domains, allowing for the rapid generation of high-quality solutions without extensive retraining. Experiments across diverse continuous control domains demonstrate that LLE-MORL consistently achieves higher Pareto front quality and efficiency than state-of-the-art approaches.