Counterfactual Programming for Optimal Control

TL;DR

This paper introduces a counterfactual optimization algorithm that enables autonomous agents to balance performance and specification costs in control tasks, addressing infeasibility and adaptability issues in complex dynamical systems.

Contribution

It presents a novel counterfactual programming approach using duality theory for autonomous trade-off tuning in optimal control problems.

Findings

Successfully balances multiple control specifications in complex systems.

Reduces reliance on expert tuning and regularization.

Enhances adaptability in dynamic and uncertain environments.

Abstract

In recent years, considerable work has been done to tackle the issue of designing control laws based on observations to allow unknown dynamical systems to perform pre-specified tasks. At least as important for autonomy, however, is the issue of learning which tasks can be performed in the first place. This is particularly critical in situations where multiple (possibly conflicting) tasks and requirements are demanded from the agent, resulting in infeasible specifications. Such situations arise due to over-specification or dynamic operating conditions and are only aggravated when the dynamical system model is learned through simulations. Often, these issues are tackled using regularization and penalties tuned based on application-specific expert knowledge. Nevertheless, this solution becomes impractical for large-scale systems, unknown operating conditions, and/or in online settings where expert input would be needed during the system operation. Instead, this work enables agents to autonomously pose, tune, and solve optimal control problems by compromising between performance and specification costs. Leveraging duality theory, it puts forward a counterfactual optimization algorithm that directly determines the specification trade-off while solving the optimal control problem.