Differentiable Spatial Planning using Transformers

TL;DR

This paper introduces Spatial Planning Transformers (SPT), a novel differentiable model for spatial path planning that leverages long-range dependencies and outperforms previous methods in manipulation and navigation tasks.

Contribution

The paper proposes SPT, a transformer-based planner that captures long-range spatial dependencies and generalizes well to unseen maps and goals, surpassing prior differentiable planners.

Findings

SPT outperforms state-of-the-art differentiable planners by 7-19%.

SPT generalizes to out-of-distribution maps and goals.

SPT effectively integrates mapping and planning in an end-to-end framework.

Abstract

We consider the problem of spatial path planning. In contrast to the classical solutions which optimize a new plan from scratch and assume access to the full map with ground truth obstacle locations, we learn a planner from the data in a differentiable manner that allows us to leverage statistical regularities from past data. We propose Spatial Planning Transformers (SPT), which given an obstacle map learns to generate actions by planning over long-range spatial dependencies, unlike prior data-driven planners that propagate information locally via convolutional structure in an iterative manner. In the setting where the ground truth map is not known to the agent, we leverage pre-trained SPTs in an end-to-end framework that has the structure of mapper and planner built into it which allows seamless generalization to out-of-distribution maps and goals. SPTs outperform prior state-of-the-art differentiable planners across all the setups for both manipulation and navigation tasks, leading to an absolute improvement of 7-19%.