iKap: Kinematics-aware Planning with Imperative Learning

TL;DR

iKap is a novel vision-to-planning system that integrates kinematic constraints into the learning process, enabling more reliable and efficient trajectory planning for robots in complex environments.

Contribution

It introduces a self-supervised, differentiable bi-level optimization framework that incorporates the robot's kinematic model into vision-based planning, improving success rates and reducing latency.

Findings

Higher success rates compared to state-of-the-art methods

Reduced planning latency

Effective integration of kinematic constraints into learning pipeline

Abstract

Trajectory planning in robotics aims to generate collision-free pose sequences that can be reliably executed. Recently, vision-to-planning systems have gained increasing attention for their efficiency and ability to interpret and adapt to surrounding environments. However, traditional modular systems suffer from increased latency and error propagation, while purely data-driven approaches often overlook the robot's kinematic constraints. This oversight leads to discrepancies between planned trajectories and those that are executable. To address these challenges, we propose iKap, a novel vision-to-planning system that integrates the robot's kinematic model directly into the learning pipeline. iKap employs a self-supervised learning approach and incorporates the state transition model within a differentiable bi-level optimization framework. This integration ensures the network learns collision-free waypoints while satisfying kinematic constraints, enabling gradient back-propagation for end-to-end training. Our experimental results demonstrate that iKap achieves higher success rates and reduced latency compared to the state-of-the-art methods. Besides the complete system, iKap offers a visual-to-planning network that seamlessly works with various controllers, providing a robust solution for robots navigating complex environments.