NMR: Neural Manifold Representation for Autonomous Driving

TL;DR

This paper introduces Neural Manifold Representation (NMR), a novel approach for autonomous driving that models the environment on a surface manifold to better handle road gradients, improving path planning accuracy.

Contribution

The paper proposes NMR, a new manifold-based representation that infers semantics and predicts waypoints, overcoming BEV limitations caused by flat-world assumptions.

Findings

NMR effectively models surface geometry for autonomous driving.

The approach improves path planning accuracy on gradient roads.

NMR demonstrates promising results on CARLA and SYNTHIA-SF datasets.

Abstract

Autonomous driving requires efficient reasoning about the Spatio-temporal nature of the semantics of the scene. Recent approaches have successfully amalgamated the traditional modular architecture of an autonomous driving stack comprising perception, prediction, and planning in an end-to-end trainable system. Such a system calls for a shared latent space embedding with interpretable intermediate trainable projected representation. One such successfully deployed representation is the Bird's-Eye View(BEV) representation of the scene in ego-frame. However, a fundamental assumption for an undistorted BEV is the local coplanarity of the world around the ego-vehicle. This assumption is highly restrictive, as roads, in general, do have gradients. The resulting distortions make path planning inefficient and incorrect. To overcome this limitation, we propose Neural Manifold Representation (NMR), a representation for the task of autonomous driving that learns to infer semantics and predict way-points on a manifold over a finite horizon, centered on the ego-vehicle. We do this using an iterative attention mechanism applied on a latent high dimensional embedding of surround monocular images and partial ego-vehicle state. This representation helps generate motion and behavior plans consistent with and cognizant of the surface geometry. We propose a sampling algorithm based on edge-adaptive coverage loss of BEV occupancy grid and associated guidance flow field to generate the surface manifold while incurring minimal computational overhead. We aim to test the efficacy of our approach on CARLA and SYNTHIA-SF.