Scaling up and Stabilizing Differentiable Planning with Implicit Differentiation

TL;DR

This paper introduces an implicit differentiation method for differentiable planning that decouples forward and backward passes, enabling scalable and stable training for large planning tasks across various domains.

Contribution

It proposes an implicit differentiation approach through the Bellman fixed-point equation, improving scalability and stability of differentiable planning methods like VIN.

Findings

Constant backward cost in planning horizon

Enhanced scalability to large tasks

Superior performance on navigation and manipulation benchmarks

Abstract

Differentiable planning promises end-to-end differentiability and adaptivity. However, an issue prevents it from scaling up to larger-scale problems: they need to differentiate through forward iteration layers to compute gradients, which couples forward computation and backpropagation, and needs to balance forward planner performance and computational cost of the backward pass. To alleviate this issue, we propose to differentiate through the Bellman fixed-point equation to decouple forward and backward passes for Value Iteration Network and its variants, which enables constant backward cost (in planning horizon) and flexible forward budget and helps scale up to large tasks. We study the convergence stability, scalability, and efficiency of the proposed implicit version of VIN and its variants and demonstrate their superiorities on a range of planning tasks: 2D navigation, visual navigation, and 2-DOF manipulation in configuration space and workspace.