Learning Granular Media Avalanche Behavior for Indirectly Manipulating Obstacles on a Granular Slope

TL;DR

This paper introduces a novel method for legged robots to manipulate obstacles on granular slopes by predicting avalanche-induced object movements using a Vision Transformer, enabling effective indirect obstacle manipulation.

Contribution

It is the first to leverage granular media avalanche dynamics for indirect obstacle manipulation on slopes using a Vision Transformer-based prediction model.

Findings

Achieved over 80% success rate in manipulation tasks with multiple obstacles.

Model accurately predicts object movements in diverse settings.

Demonstrated generalization to objects with different physics properties.

Abstract

Legged robot locomotion on sand slopes is challenging due to the complex dynamics of granular media and how the lack of solid surfaces can hinder locomotion. A promising strategy, inspired by ghost crabs and other organisms in nature, is to strategically interact with rocks, debris, and other obstacles to facilitate movement. To provide legged robots with this ability, we present a novel approach that leverages avalanche dynamics to indirectly manipulate objects on a granular slope. We use a Vision Transformer (ViT) to process image representations of granular dynamics and robot excavation actions. The ViT predicts object movement, which we use to determine which leg excavation action to execute. We collect training data from 100 real physical trials and, at test time, deploy our trained model in novel settings. Experimental results suggest that our model can accurately predict object movements and achieve a success rate $\geq 80\%$ in a variety of manipulation tasks with up to four obstacles, and can also generalize to objects with different physics properties. To our knowledge, this is the first paper to leverage granular media avalanche dynamics to indirectly manipulate objects on granular slopes. Supplementary material is available at https://sites.google.com/view/grain-corl2024/home.