TL;DR
This paper introduces MPC^2, a hierarchical model-based control algorithm that enables zero-shot and near-real-time control of high-dimensional musculoskeletal systems, reducing manual tuning and improving robustness.
Contribution
The paper presents MPC^2, a novel hierarchical control method combining sampling-based predictive control with morphology-aware proportional control for complex dynamical systems.
Findings
Successfully controls high-dimensional musculoskeletal models in various tasks
Reduces need for manual reward engineering through black-box tuning
Achieves near-real-time control in complex motion scenarios
Abstract
Controlling high-dimensional nonlinear systems, such as those found in biological and robotic applications, is challenging due to large state and action spaces. While deep reinforcement learning has achieved a number of successes in these domains, it is computationally intensive and time consuming, and therefore not suitable for solving large collections of tasks that require significant manual tuning. In this work, we introduce Model Predictive Control with Morphology-aware Proportional Control (MPC^2), a hierarchical model-based learning algorithm for zero-shot and near-real-time control of high-dimensional complex dynamical systems. MPC^2 uses a sampling-based model predictive controller for target posture planning, and enables robust control for high-dimensional tasks by incorporating a morphology-aware proportional controller for actuator coordination. The algorithm enables motion…
Peer Reviews
Decision·ICLR 2025 Poster
- The presented method is simple, can handle the high dimensionality of the action space associated with muscle-actuated systems, and performs well on several challenging tasks, such as walking on rough terrain, walking on slopes, or over stairs. - Fast cost-design iteration time to evaluate/generate new controllers compared to Deep RL methods. - The paper is well-written overall and the ablation studies justify different parts of the proposed method.
- I have concerns that the conference might not be a good fit for the paper. I struggle to classify the presented hierarchical sampling-based approach for control as an area or subfield of machine learning. Especially, considering that MPPI and many of its variants have been traditionally presented in robotics conferences. If the authors can clarify, how the conference is a good fit for the paper or if this is not a concern for any other reviewer, I will not oppose the acceptance of this paper i
The paper definitely presents some key strengths: 1. Originality: While breaking down the high-dimensional control/planning problem into multiple stages is a very know concept, the paper presented first of its kind to integrate MPPI with Morphology Aware controller. The paper thus provides a new solution for high dimensional motion control. 2. Quality: The paper provides a thorough experimental validation in Mujoco simulation. On technical aspect, the use of Bayesian optimization for automated c
I found several weak points in relation to the experiments and validation of the proposed method (it could be also a result of my background from classical control theory). 1. Lack of quantitative stability metrics: The paper lacks in presenting the performance of their approach on key stability metrics used across the humanoids community like centre of mass polygon support, energy efficiency, etc. Additionally, integrating these metrics into the MPC cost function could enhance the results sig
The paper shows that a hierarchical and sampling-based implementation of a model-based predictive controller (MPC) can control in simulation a high-dimensional musculoskeletal system for the first time. This is to be contrasted with DRL methods that take a very long time to optimize policies, moreover they need to typically be re-run for each task. In that sense there's a clear contribution to the simulation literature on how to control such high dimensional systems.
I think the paper has some good contributions to control and simulation of high-dimensional systems as mentioned above, yet there are several weaknesses (some of which can be addressed in the rebuttal phase hopefully): - There is a very extensive literature on MPC algorithms and analysis, yet the paper does not compare the proposed approach to any other MPC algorithm. Would all of the MPC methods proposed in the literature fail for this complex dynamics case? It is not clear from the text. The
Videos
