Hidden Footprints: Learning Contextual Walkability from 3D Human Trails

TL;DR

This paper introduces a novel method for predicting walkable areas in scenes by leveraging 3D information and augmented data, improving walkability prediction without additional labeling.

Contribution

It proposes a data augmentation technique using hidden footprints and a specialized training strategy for sparse labels to enhance walkability prediction models.

Findings

Outperforms baseline models on Waymo and Cityscapes datasets.

Effective augmentation of walkable regions using 3D information.

Robust prediction of walkability from a single image.

Abstract

Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis. Yet learning a computational model for this purpose is challenging due to semantic ambiguity and a lack of labeled data: current datasets only tell you where people are, not where they could be. We tackle this problem by leveraging information from existing datasets, without additional labeling. We first augment the set of valid, labeled walkable regions by propagating person observations between images, utilizing 3D information to create what we call hidden footprints. However, this augmented data is still sparse. We devise a training strategy designed for such sparse labels, combining a class-balanced classification loss with a contextual adversarial loss. Using this strategy, we demonstrate a model that learns to predict a walkability map from a single image. We evaluate our model on the Waymo and Cityscapes datasets, demonstrating superior performance compared to baselines and state-of-the-art models.