Robot Data Curation with Mutual Information Estimators

TL;DR

This paper introduces a novel mutual information estimator using k-nearest neighbors and VAE embeddings to assess and improve the quality of demonstration datasets for robotic imitation learning, leading to better policy performance.

Contribution

The paper presents a new mutual information estimation technique tailored for robotics data, enabling quality assessment and filtering of demonstrations to enhance learning outcomes.

Findings

Effective dataset partitioning by demonstration quality

Improved policy performance with data filtering

Applicable to both simulation and real-world environments

Abstract

The performance of imitation learning policies often hinges on the datasets with which they are trained. Consequently, investment in data collection for robotics has grown across both industrial and academic labs. However, despite the marked increase in the quantity of demonstrations collected, little work has sought to assess the quality of said data despite mounting evidence of its importance in other areas such as vision and language. In this work, we take a critical step towards addressing the data quality in robotics. Given a dataset of demonstrations, we aim to estimate the relative quality of individual demonstrations in terms of both action diversity and predictability. To do so, we estimate the average contribution of a trajectory towards the mutual information between states and actions in the entire dataset, which captures both the entropy of the marginal action distribution and the state-conditioned action entropy. Though commonly used mutual information estimators require vast amounts of data often beyond the scale available in robotics, we introduce a novel technique based on k-nearest neighbor estimates of mutual information on top of simple VAE embeddings of states and actions. Empirically, we demonstrate that our approach is able to partition demonstration datasets by quality according to human expert scores across a diverse set of benchmarks spanning simulation and real world environments. Moreover, training policies based on data filtered by our method leads to a 5-10% improvement in RoboMimic and better performance on real ALOHA and Franka setups.