Visibility Maximization Controller for Robotic Manipulation

TL;DR

This paper introduces a reactive, optimization-based controller for robotic manipulation that reduces self-occlusions in eye-to-hand camera setups, balancing visibility with task success and efficiency.

Contribution

It presents a novel controller that encodes line-of-sight visibility as a soft constraint, enabling coordinated control of robot base, arm, and head to minimize occlusions.

Findings

Improves visibility to target objects without significantly reducing task success.

Outperforms planning-based methods in efficiency and success rate.

Demonstrated in both simulated and real-world experiments with various robot configurations.

Abstract

Occlusions caused by a robot's own body is a common problem for closed-loop control methods employed in eye-to-hand camera setups. We propose an optimization-based reactive controller that minimizes self-occlusions while achieving a desired goal pose. The approach allows coordinated control between the robot's base, arm and head by encoding the line-of-sight visibility to the target as a soft constraint along with other task-related constraints, and solving for feasible joint and base velocities. The generalizability of the approach is demonstrated in simulated and real-world experiments, on robots with fixed or mobile bases, with moving or fixed objects, and multiple objects. The experiments revealed a trade-off between occlusion rates and other task metrics. While a planning-based baseline achieved lower occlusion rates than the proposed controller, it came at the expense of highly inefficient paths and a significant drop in the task success. On the other hand, the proposed controller is shown to improve visibility to the line target object(s) without sacrificing too much from the task success and efficiency. Videos and code can be found at: rhys-newbury.github.io/projects/vmc/.