Monolithic vs. hybrid controller for multi-objective Sim-to-Real learning

TL;DR

This paper compares monolithic and hybrid controllers for multi-objective robotic tasks in Sim-to-Real learning, finding hybrid controllers easier to train and more effective in success rates.

Contribution

It introduces a comparative analysis of monolithic versus hybrid controllers for multi-objective Sim-to-Real robotic tasks, highlighting the advantages of hybrid approaches.

Findings

Hybrid controllers are easier to train than monolithic ones.

Hybrid controllers achieve better success-failure trade-offs.

Simulated controllers transfer effectively to real-world setups.

Abstract

Simulation to real (Sim-to-Real) is an attractive approach to construct controllers for robotic tasks that are easier to simulate than to analytically solve. Working Sim-to-Real solutions have been demonstrated for tasks with a clear single objective such as "reach the target". Real world applications, however, often consist of multiple simultaneous objectives such as "reach the target" but "avoid obstacles". A straightforward solution in the context of reinforcement learning (RL) is to combine multiple objectives into a multi-term reward function and train a single monolithic controller. Recently, a hybrid solution based on pre-trained single objective controllers and a switching rule between them was proposed. In this work, we compare these two approaches in the multi-objective setting of a robot manipulator to reach a target while avoiding an obstacle. Our findings show that the training of a hybrid controller is easier and obtains a better success-failure trade-off than a monolithic controller. The controllers trained in simulator were verified by a real set-up.