CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC

TL;DR

This paper introduces a new benchmark and sequential policies for high-dimensional dynamic algorithm configuration, effectively managing interdependent action dimensions with varying importance, and demonstrates their superior performance over traditional methods.

Contribution

The paper presents a novel benchmark for CANDID DAC problems and proposes sequential policies that better handle interdependencies and importance differences in high-dimensional action spaces.

Findings

Sequential policies outperform independent policies in CANDID spaces.

Sequential policies mitigate exponential growth in action space complexity.

Proposed methods overcome scalability issues of single-policy approaches.

Abstract

High-dimensional action spaces remain a challenge for dynamic algorithm configuration (DAC). Interdependencies and varying importance between action dimensions are further known key characteristics of DAC problems. We argue that these Coupled Action Dimensions with Importance Differences (CANDID) represent aspects of the DAC problem that are not yet fully explored. To address this gap, we introduce a new white-box benchmark within the DACBench suite that simulates the properties of CANDID. Further, we propose sequential policies as an effective strategy for managing these properties. Such policies factorize the action space and mitigate exponential growth by learning a policy per action dimension. At the same time, these policies accommodate the interdependence of action dimensions by fostering implicit coordination. We show this in an experimental study of value-based policies on our new benchmark. This study demonstrates that sequential policies significantly outperform independent learning of factorized policies in CANDID action spaces. In addition, they overcome the scalability limitations associated with learning a single policy across all action dimensions. The code used for our experiments is available under https://github.com/PhilippBordne/candidDAC.