Predicting Chronic Homelessness: The Importance of Comparing Algorithms using Client Histories

TL;DR

This study compares simple threshold methods with complex machine learning algorithms for predicting chronic homelessness, finding similar candidate groups despite differences in traditional performance metrics, highlighting practical implications for resource-limited organizations.

Contribution

It demonstrates that simple threshold methods can identify similar candidate groups as advanced machine learning models, offering a practical alternative for resource-constrained settings.

Findings

Machine learning algorithms outperform threshold methods on standard metrics.

All methods identify similar groups of potential candidates.

Simple threshold approach requires less technological infrastructure.

Abstract

This paper investigates how to best compare algorithms for predicting chronic homelessness for the purpose of identifying good candidates for housing programs. Predictive methods can rapidly refer potentially chronic shelter users to housing but also sometimes incorrectly identify individuals who will not become chronic (false positives). We use shelter access histories to demonstrate that these false positives are often still good candidates for housing. Using this approach, we compare a simple threshold method for predicting chronic homelessness to the more complex logistic regression and neural network algorithms. While traditional binary classification performance metrics show that the machine learning algorithms perform better than the threshold technique, an examination of the shelter access histories of the cohorts identified by the three algorithms show that they select groups with very similar characteristics. This has important implications for resource constrained not-for-profit organizations since the threshold technique can be implemented using much simpler information technology infrastructure than the machine learning algorithms.