Adaptive Visual Imitation Learning for Robotic Assisted Feeding Across Varied Bowl Configurations and Food Types

TL;DR

This paper presents AVIL, a visual imitation learning framework with spatial attention for robotic feeding, demonstrating robustness and adaptability across various bowl setups and food types, with significant success rate improvements.

Contribution

Introduces AVIL, a novel adaptive visual imitation learning framework that generalizes food scooping tasks across diverse scenarios with minimal training data.

Findings

Up to 2.5x success rate improvement over baseline

Effective zero-shot generalization to new bowl configurations and food types

Robust performance across granular, semi-solid, and liquid foods

Abstract

In this study, we introduce a novel visual imitation network with a spatial attention module for robotic assisted feeding (RAF). The goal is to acquire (i.e., scoop) food items from a bowl. However, achieving robust and adaptive food manipulation is particularly challenging. To deal with this, we propose a framework that integrates visual perception with imitation learning to enable the robot to handle diverse scenarios during scooping. Our approach, named AVIL (adaptive visual imitation learning), exhibits adaptability and robustness across different bowl configurations in terms of material, size, and position, as well as diverse food types including granular, semi-solid, and liquid, even in the presence of distractors. We validate the effectiveness of our approach by conducting experiments on a real robot. We also compare its performance with a baseline. The results demonstrate improvement over the baseline across all scenarios, with an enhancement of up to 2.5 times in terms of a success metric. Notably, our model, trained solely on data from a transparent glass bowl containing granular cereals, showcases generalization ability when tested zero-shot on other bowl configurations with different types of food.