StreetLens: Enabling Human-Centered AI Agents for Neighborhood Assessment from Street View Imagery

TL;DR

StreetLens is a configurable AI workflow that combines vision language models with social science expertise to automate and scale neighborhood assessments from street view imagery, improving flexibility and robustness.

Contribution

It introduces a human-centered, domain-informed VLM framework for neighborhood analysis that adapts to diverse research needs and integrates prior survey data.

Findings

Automates neighborhood characteristic assessments from street view images.

Supports subjective and objective environmental annotations.

Enhances scalability and adaptability of neighborhood studies.

Abstract

Traditionally, neighborhood studies have used interviews, surveys, and manual image annotation guided by detailed protocols to identify environmental characteristics, including physical disorder, decay, street safety, and sociocultural symbols, and to examine their impact on developmental and health outcomes. Although these methods yield rich insights, they are time-consuming and require intensive expert intervention. Recent technological advances, including vision language models (VLMs), have begun to automate parts of this process; however, existing efforts are often ad hoc and lack adaptability across research designs and geographic contexts. In this paper, we present StreetLens, a user-configurable human-centered workflow that integrates relevant social science expertise into a VLM for scalable neighborhood environmental assessments. StreetLens mimics the process of trained human coders by focusing the analysis on questions derived from established interview protocols, retrieving relevant street view imagery (SVI), and generating a wide spectrum of semantic annotations from objective features (e.g., the number of cars) to subjective perceptions (e.g., the sense of disorder in an image). By enabling researchers to define the VLM's role through domain-informed prompting, StreetLens places domain knowledge at the core of the analysis process. It also supports the integration of prior survey data to enhance robustness and expand the range of characteristics assessed in diverse settings. StreetLens represents a shift toward flexible and agentic AI systems that work closely with researchers to accelerate and scale neighborhood studies. StreetLens is publicly available at https://knowledge-computing.github.io/projects/streetlens.