Distributional Multi-Objective Decision Making

TL;DR

This paper introduces a distributional approach to multi-objective decision making, defining new dominance criteria and sets that better identify optimal policies for risk-averse decision makers, with efficient algorithms and experimental validation.

Contribution

It proposes the distributional undominated set and convex distributional undominated set, along with algorithms to compute and prune these sets for improved decision support.

Findings

The distributional undominated set includes policies ignored by Pareto front.

The convex distributional undominated set maximizes expected utility for risk-averse agents.

Experimental results demonstrate the methods' effectiveness and practicality.

Abstract

For effective decision support in scenarios with conflicting objectives, sets of potentially optimal solutions can be presented to the decision maker. We explore both what policies these sets should contain and how such sets can be computed efficiently. With this in mind, we take a distributional approach and introduce a novel dominance criterion relating return distributions of policies directly. Based on this criterion, we present the distributional undominated set and show that it contains optimal policies otherwise ignored by the Pareto front. In addition, we propose the convex distributional undominated set and prove that it comprises all policies that maximise expected utility for multivariate risk-averse decision makers. We propose a novel algorithm to learn the distributional undominated set and further contribute pruning operators to reduce the set to the convex distributional undominated set. Through experiments, we demonstrate the feasibility and effectiveness of these methods, making this a valuable new approach for decision support in real-world problems.