An Introduction to Zero-Order Optimization Techniques for Robotics

TL;DR

This paper provides a mathematical tutorial on zero-order optimization, unifying various algorithms used in robotics, especially for trajectory and policy optimization, and introduces new reinforcement learning algorithms.

Contribution

It offers a unifying framework for zero-order methods in robotics and derives novel RL algorithms based on this perspective.

Findings

Classified many trajectory optimization methods under a common framework

Derived new competitive reinforcement learning algorithms

Provided insights into the advantages of zero-order techniques in robotics

Abstract

Zero-order optimization techniques are becoming increasingly popular in robotics due to their ability to handle non-differentiable functions and escape local minima. These advantages make them particularly useful for trajectory optimization and policy optimization. In this work, we propose a mathematical tutorial on random search. It offers a simple and unifying perspective for understanding a wide range of algorithms commonly used in robotics. Leveraging this viewpoint, we classify many trajectory optimization methods under a common framework and derive novel competitive RL algorithms.