SideSeeing: A multimodal dataset and collection of tools for sidewalk assessment

TL;DR

SideSeeing introduces a comprehensive multimodal dataset and tools for sidewalk assessment, combining video and sensor data to facilitate urban accessibility evaluations.

Contribution

The paper presents a new dataset and framework for sidewalk environment analysis, integrating synchronized video and sensor data with a novel taxonomy for scene identification.

Findings

Collected 12 km of sidewalk data with 325,000 video frames and sensor data.

Developed a new taxonomy for sidewalk scene identification.

Provided publicly available tools and dataset for urban accessibility studies.

Abstract

This paper introduces SideSeeing, a novel initiative that provides tools and datasets for assessing the built environment. We present a framework for street-level data acquisition, loading, and analysis. Using the framework, we collected a novel dataset that integrates synchronized video footaged captured from chest-mounted mobile devices with sensor data (accelerometer, gyroscope, magnetometer, and GPS). Each data sample represents a path traversed by a user filming sidewalks near hospitals in Brazil and the USA. The dataset encompasses three hours of content covering 12 kilometers around nine hospitals, and includes 325,000 video frames with corresponding sensor data. Additionally, we present a novel 68-element taxonomy specifically created for sidewalk scene identification. SideSeeing is a step towards a suite of tools that urban experts can use to perform in-depth sidewalk accessibility evaluations. SideSeeing data and tools are publicly available at https://sites.usp.br/sideseeing/.