MonoForce: Self-supervised Learning of Physics-informed Model for Predicting Robot-terrain Interaction

TL;DR

MonoForce introduces a physics-informed, self-supervised learning model that predicts robot-terrain interactions from images, excelling especially on deformable terrains like tall grass, enhancing autonomous navigation capabilities.

Contribution

It presents a novel, explainable, end-to-end differentiable model combining black-box force prediction with a white-box physics module for deformable terrain navigation.

Findings

Comparable accuracy to state-of-the-art on rigid terrain

Superior accuracy on non-rigid terrain like tall grass

Reproducible code and datasets released

Abstract

While autonomous navigation of mobile robots on rigid terrain is a well-explored problem, navigating on deformable terrain such as tall grass or bushes remains a challenge. To address it, we introduce an explainable, physics-aware and end-to-end differentiable model which predicts the outcome of robot-terrain interaction from camera images, both on rigid and non-rigid terrain. The proposed MonoForce model consists of a black-box module which predicts robot-terrain interaction forces from onboard cameras, followed by a white-box module, which transforms these forces and a control signals into predicted trajectories, using only the laws of classical mechanics. The differentiable white-box module allows backpropagating the predicted trajectory errors into the black-box module, serving as a self-supervised loss that measures consistency between the predicted forces and ground-truth trajectories of the robot. Experimental evaluation on a public dataset and our data has shown that while the prediction capabilities are comparable to state-of-the-art algorithms on rigid terrain, MonoForce shows superior accuracy on non-rigid terrain such as tall grass or bushes. To facilitate the reproducibility of our results, we release both the code and datasets.