How to Build Robust, Scalable Models for GSV-Based Indicators in Neighborhood Research

TL;DR

This paper investigates how to adapt and select computer vision models for analyzing Google Street View images in neighborhood health research, focusing on transferability, unsupervised training, and practical performance improvements.

Contribution

It provides empirical insights and practical guidelines for adapting foundation models to GSV imagery, especially with limited labeled data and the use of unsupervised training strategies.

Findings

Unsupervised adaptation improves model performance on GSV data.

Model selection depends on dataset size and label availability.

Transfer learning from ImageNet has limitations for GSV imagery.

Abstract

A substantial body of health research demonstrates a strong link between neighborhood environments and health outcomes. Recently, there has been increasing interest in leveraging advances in computer vision to enable large-scale, systematic characterization of neighborhood built environments. However, the generalizability of vision models across fundamentally different domains remains uncertain, for example, transferring knowledge from ImageNet to the distinct visual characteristics of Google Street View (GSV) imagery. In applied fields such as social health research, several critical questions arise: which models are most appropriate, whether to adopt unsupervised training strategies, what training scale is feasible under computational constraints, and how much such strategies benefit downstream performance. These decisions are often costly and require specialized expertise. In this paper, we answer these questions through empirical analysis and provide practical insights into how to select and adapt foundation models for datasets with limited size and labels, while leveraging larger, unlabeled datasets through unsupervised training. Our study includes comprehensive quantitative and visual analyses comparing model performance before and after unsupervised adaptation.