Adaptive t-Momentum-based Optimization for Unknown Ratio of Outliers in Amateur Data in Imitation Learning

TL;DR

This paper introduces an adaptive t-momentum optimization algorithm that enhances behavioral cloning by effectively handling outliers and imperfect demonstrations in imitation learning, leading to more robust robot skill transfer.

Contribution

The paper proposes an adaptive t-momentum optimizer based on Student's t-distribution, improving robustness in behavioral cloning with unknown outlier ratios.

Findings

Robust BC imitators perform well despite noisy demonstrations

The algorithm adapts automatically to different outlier levels

Enhanced data utilization with reduced adverse effects

Abstract

Behavioral cloning (BC) bears a high potential for safe and direct transfer of human skills to robots. However, demonstrations performed by human operators often contain noise or imperfect behaviors that can affect the efficiency of the imitator if left unchecked. In order to allow the imitators to effectively learn from imperfect demonstrations, we propose to employ the robust t-momentum optimization algorithm. This algorithm builds on the Student's t-distribution in order to deal with heavy-tailed data and reduce the effect of outlying observations. We extend the t-momentum algorithm to allow for an adaptive and automatic robustness and show empirically how the algorithm can be used to produce robust BC imitators against datasets with unknown heaviness. Indeed, the imitators trained with the t-momentum-based Adam optimizers displayed robustness to imperfect demonstrations on two different manipulation tasks with different robots and revealed the capability to take advantage of the additional data while reducing the adverse effect of non-optimal behaviors.