CMAX++ : Leveraging Experience in Planning and Execution using Inaccurate Models

TL;DR

CMAX++ enhances robotic planning by integrating experience-based learning with model-based planning, improving solution quality over repetitions despite inaccuracies in the dynamical models.

Contribution

It introduces CMAX++, a novel method that combines model-free learning with model-based planning to iteratively improve robotic task planning in uncertain environments.

Findings

Outperforms baselines in simulated navigation tasks with model inaccuracies.

Guarantees asymptotic convergence to optimal path cost.

Effective in 3D navigation and 7D pick-and-place tasks with unknown parameters.

Abstract

Given access to accurate dynamical models, modern planning approaches are effective in computing feasible and optimal plans for repetitive robotic tasks. However, it is difficult to model the true dynamics of the real world before execution, especially for tasks requiring interactions with objects whose parameters are unknown. A recent planning approach, CMAX, tackles this problem by adapting the planner online during execution to bias the resulting plans away from inaccurately modeled regions. CMAX, while being provably guaranteed to reach the goal, requires strong assumptions on the accuracy of the model used for planning and fails to improve the quality of the solution over repetitions of the same task. In this paper we propose CMAX++, an approach that leverages real-world experience to improve the quality of resulting plans over successive repetitions of a robotic task. CMAX++ achieves this by integrating model-free learning using acquired experience with model-based planning using the potentially inaccurate model. We provide provable guarantees on the completeness and asymptotic convergence of CMAX++ to the optimal path cost as the number of repetitions increases. CMAX++ is also shown to outperform baselines in simulated robotic tasks including 3D mobile robot navigation where the track friction is incorrectly modeled, and a 7D pick-and-place task where the mass of the object is unknown leading to discrepancy between true and modeled dynamics.