Using Images as Covariates: Measuring Curb Appeal with Deep Learning

TL;DR

This paper introduces a novel approach that integrates image data into econometric models using deep learning, significantly enhancing the accuracy of real estate price forecasts by leveraging visual features as covariates.

Contribution

The study develops a method to encode images with deep learning classifiers and segmentation, combining these features with traditional data to improve predictive accuracy in real estate valuation.

Findings

Image-based forecasts improve out-of-sample prediction accuracy.

Encoded visual features correlate with key property and location covariates.

Combining image data with standard models enhances forecasting performance.

Abstract

This paper details an innovative methodology to integrate image data into traditional econometric models. Motivated by forecasting sales prices for residential real estate, we harness the power of deep learning to add "information" contained in images as covariates. Specifically, images of homes were categorized and encoded using an ensemble of image classifiers (ResNet-50, VGG16, MobileNet, and Inception V3). Unique features presented within each image were further encoded through panoptic segmentation. Forecasts from a neural network trained on the encoded data results in improved out-of-sample predictive power. We also combine these image-based forecasts with standard hedonic real estate property and location characteristics, resulting in a unified dataset. We show that image-based forecasts increase the accuracy of hedonic forecasts when encoded features are regarded as additional covariates. We also attempt to "explain" which covariates the image-based forecasts are most highly correlated with. The study exemplifies the benefits of interdisciplinary methodologies, merging machine learning and econometrics to harness untapped data sources for more accurate forecasting.