DART: Learning-Enhanced Model Predictive Control for Dual-Arm Non-Prehensile Manipulation

TL;DR

DART introduces a dual-arm manipulation framework combining nonlinear MPC with various object dynamics models, enabling precise non-prehensile object handling on a tray in simulation, with a focus on model trade-offs.

Contribution

It is the first framework for non-prehensile dual-arm manipulation of objects on a tray, integrating multiple dynamics modeling strategies within MPC.

Findings

Reinforcement learning model generalizes across object properties.

Physics-based model offers fast response with less accuracy.

Online regression adapts to changing object dynamics.

Abstract

What appears effortless to a human waiter remains a major challenge for robots. Manipulating objects nonprehensilely on a tray is inherently difficult, and the complexity is amplified in dual-arm settings. Such tasks are highly relevant to service robotics in domains such as hotels and hospitality, where robots must transport and reposition diverse objects with precision. We present DART, a novel dual-arm framework that integrates nonlinear Model Predictive Control (MPC) with an optimization-based impedance controller to achieve accurate object motion relative to a dynamically controlled tray. The framework systematically evaluates three complementary strategies for modeling tray-object dynamics as the state transition function within our MPC formulation: (i) a physics-based analytical model, (ii) an online regression based identification model that adapts in real-time, and (iii) a reinforcement learning-based dynamics model that generalizes across object properties. Our pipeline is validated in simulation with objects of varying mass, geometry, and friction coefficients. Extensive evaluations highlight the trade-offs among the three modeling strategies in terms of settling time, steady-state error, control effort, and generalization across objects. To the best of our knowledge, DART constitutes the first framework for non-prehensile dual-arm manipulation of objects on a tray. Project Link: https://dart-icra.github.io/dart/