Necessary and Sufficient Conditions for the Optimization-Based Concurrent Execution of Learned Robotic Tasks

TL;DR

This paper establishes necessary and sufficient conditions for the concurrent execution of multiple learned RL tasks using an optimization framework, enhancing understanding and applicability of such methods.

Contribution

It provides fundamental theorems characterizing when learned value functions can be executed concurrently, and extends the framework to include discounted value functions.

Findings

Theorems specify conditions for concurrent execution of learned tasks.

Framework extended to handle discounted value functions.

Insights into inherent and achievable concurrency of learned control tasks.

Abstract

In this work, we consider the problem of executing multiple tasks encoded by value functions, each learned through Reinforcement Learning, using an optimization-based framework. Prior works develop such a framework, but left unanswered a fundamental question of when learned value functions can be concurrently executed. The main contribution of this work is to present theorems which provide necessary and sufficient conditions to concurrently execute sets of learned tasks within subsets of the state space, using a previously proposed min-norm controller. These theorems provide insight into when learned control tasks are possible to be made concurrently executable, when they might already inherently be concurrently executable and when it is not possible at all to make a set of learned tasks concurrently executable using the previously proposed methods. Additional contributions of this work include extending the optimization-based framework to execute multiple tasks encoded by value functions to also account for value functions trained with a discount factor, making the overall framework more compatible with standard RL practices.