Trajectory-Constrained Deep Latent Visual Attention for Improved Local Planning in Presence of Heterogeneous Terrain

TL;DR

This paper introduces a novel deep learning approach that uses trajectory-constrained visual attention in latent space to improve local planning for visual navigation across heterogeneous terrains, enhancing generalization and efficiency.

Contribution

The method uniquely integrates trajectory-constrained visual attention in latent space with joint optimization, improving local planning in diverse terrains compared to existing attention mechanisms.

Findings

Enhanced generalization over baseline methods

Improved learning efficiency in navigation tasks

Effective in both simulated and real-world environments

Abstract

We present a reward-predictive, model-based deep learning method featuring trajectory-constrained visual attention for local planning in visual navigation tasks. Our method learns to place visual attention at locations in latent image space which follow trajectories caused by vehicle control actions to enhance predictive accuracy during planning. The attention model is jointly optimized by the task-specific loss and an additional trajectory-constraint loss, allowing adaptability yet encouraging a regularized structure for improved generalization and reliability. Importantly, visual attention is applied in latent feature map space instead of raw image space to promote efficient planning. We validated our model in visual navigation tasks of planning low turbulence, collision-free trajectories in off-road settings and hill climbing with locking differentials in the presence of slippery terrain. Experiments involved randomized procedural generated simulation and real-world environments. We found our method improved generalization and learning efficiency when compared to no-attention and self-attention alternatives.