Learning Objective Functions Incrementally by Inverse Optimal Control

TL;DR

This paper introduces an inverse optimal control approach that incrementally learns a robot's control objective function from evolving collections of trajectory segments, enabling adaptive learning from limited demonstrations.

Contribution

It presents a novel incremental learning method that updates the objective function using informative trajectory segments, applicable to complex robotic systems.

Findings

Effective on simulated robot arm with small segments

Successful on quadrotor system with limited demonstrations

Incremental learning improves adaptability and accuracy

Abstract

This paper proposes an inverse optimal control method which enables a robot to incrementally learn a control objective function from a collection of trajectory segments. By saying incrementally, it means that the collection of trajectory segments is enlarged because additional segments are provided as time evolves. The unknown objective function is parameterized as a weighted sum of features with unknown weights. Each trajectory segment is a small snippet of optimal trajectory. The proposed method shows that each trajectory segment, if informative, can pose a linear constraint to the unknown weights, thus, the objective function can be learned by incrementally incorporating all informative segments. Effectiveness of the method is shown on a simulated 2-link robot arm and a 6-DoF maneuvering quadrotor system, in each of which only small demonstration segments are available.