KARL: Kalman-Filter Assisted Reinforcement Learner for Dynamic Object Tracking and Grasping

TL;DR

KARL is a novel reinforcement learning framework that integrates a Kalman filter and a multi-stage curriculum to improve dynamic object tracking and grasping in realistic environments, demonstrating superior performance over previous methods.

Contribution

The paper introduces a new RL-based system with a Kalman filter layer and a six-stage curriculum, significantly enhancing grasping capabilities and robustness in dynamic scenarios.

Findings

Higher grasp success rates in real-world tests

Faster robot execution speeds

Effective tracking despite occlusions and rapid movements

Abstract

We present Kalman-filter Assisted Reinforcement Learner (KARL) for dynamic object tracking and grasping over eye-on-hand (EoH) systems, significantly expanding such systems capabilities in challenging, realistic environments. In comparison to the previous state-of-the-art, KARL (1) incorporates a novel six-stage RL curriculum that doubles the system's motion range, thereby greatly enhancing the system's grasping performance, (2) integrates a robust Kalman filter layer between the perception and reinforcement learning (RL) control modules, enabling the system to maintain an uncertain but continuous 6D pose estimate even when the target object temporarily exits the camera's field-of-view or undergoes rapid, unpredictable motion, and (3) introduces mechanisms to allow retries to gracefully recover from unavoidable policy execution failures. Extensive evaluations conducted in both simulation and real-world experiments qualitatively and quantitatively corroborate KARL's advantage over earlier systems, achieving higher grasp success rates and faster robot execution speed. Source code and supplementary materials for KARL will be made available at: https://github.com/arc-l/karl.