PathwayBench: Assessing Routability of Pedestrian Pathway Networks Inferred from Multi-City Imagery

TL;DR

This paper introduces PathwayBench, a comprehensive benchmark dataset and evaluation framework for assessing the routability of pedestrian pathway networks extracted from aerial imagery, addressing a critical gap in urban mobility applications.

Contribution

It provides the largest manually annotated dataset covering 3,000 km^2 across 8 cities and proposes new metrics focused on routability and utility for evaluating pathway extraction methods.

Findings

Benchmark reveals strengths and weaknesses of existing methods.

Local routability measures serve as proxies for global routability.

Evaluation metrics improve assessment of pathway extraction quality.

Abstract

Applications to support pedestrian mobility in urban areas require a complete, and routable graph representation of the built environment. Globally available information, including aerial imagery provides a scalable source for constructing these path networks, but the associated learning problem is challenging: Relative to road network pathways, pedestrian network pathways are narrower, more frequently disconnected, often visually and materially variable in smaller areas, and their boundaries are broken up by driveway incursions, alleyways, marked or unmarked crossings through roadways. Existing algorithms to extract pedestrian pathway network graphs are inconsistently evaluated and tend to ignore routability, making it difficult to assess utility for mobility applications: Even if all path segments are available, discontinuities could dramatically and arbitrarily shift the overall path taken by a pedestrian. In this paper, we describe a first standard benchmark for the pedestrian pathway graph extraction problem, comprising the largest available dataset equipped with manually vetted ground truth annotations (covering $3,000 km^2$ land area in regions from 8 cities), and a family of evaluation metrics centering routability and downstream utility. By partitioning the data into polygons at the scale of individual intersections, we compute local routability as an efficient proxy for global routability. We consider multiple measures of polygon-level routability and compare predicted measures with ground truth to construct evaluation metrics. Using these metrics, we show that this benchmark can surface strengths and weaknesses of existing methods that are hidden by simple edge-counting metrics over single-region datasets used in prior work, representing a challenging, high-impact problem in computer vision and machine learning.