Household poverty classification in data-scarce environments: a machine learning approach

TL;DR

This paper presents a machine learning method for classifying household poverty in data-scarce environments using a simple, cost-effective survey-based scorecard that generalizes across sub-national regions.

Contribution

It introduces a scalable, low-cost poverty prediction model leveraging standard statistical techniques and minimal questions, suitable for diverse regions within a country.

Findings

Model accurately predicts poverty using only ten questions.

Out-of-sample predictions successfully distinguish poor households.

Framework is practical for low-resource settings with sub-national diversity.

Abstract

We describe a method to identify poor households in data-scarce countries by leveraging information contained in nationally representative household surveys. It employs standard statistical learning techniques---cross-validation and parameter regularization---which together reduce the extent to which the model is over-fitted to match the idiosyncracies of observed survey data. The automated framework satisfies three important constraints of this development setting: i) The prediction model uses at most ten questions, which limits the costs of data collection; ii) No computation beyond simple arithmetic is needed to calculate the probability that a given household is poor, immediately after data on the ten indicators is collected; and iii) One specification of the model (i.e. one scorecard) is used to predict poverty throughout a country that may be characterized by significant sub-national differences. Using survey data from Zambia, the model's out-of-sample predictions distinguish poor households from non-poor households using information contained in ten questions.