Nonsmooth optimal value and policy functions in mechanical systems subject to unilateral constraints

TL;DR

This paper demonstrates that in contact-rich mechanical systems, optimal value and policy functions are inherently nonsmooth, challenging the effectiveness of traditional smooth approximation and gradient-based methods in such contexts.

Contribution

It reveals the fundamental nonsmooth nature of value and policy functions in contact-rich mechanical systems, highlighting limitations of existing smooth optimization approaches.

Findings

Value and policy functions are generally nonsmooth in contact-rich systems.

Traditional smooth approximation methods may not be suitable for such systems.

Implications for designing control algorithms for robots with contact dynamics.

Abstract

State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in optimal control of mechanical systems subject to unilateral constraints -- i.e. the contact-rich dynamics of robot locomotion and manipulation -- are generally nonsmooth due to the underlying dynamics exhibiting discontinuous or piecewise-differentiable trajectory outcomes. Simple mechanical systems are used to illustrate this result and the implications for optimal control of contact-rich robot dynamics.