A Comparative Analysis of Wealth Index Predictions in Africa between three Multi-Source Inference Models

TL;DR

This paper compares three multi-source inference models for predicting wealth indices in Africa, highlighting discrepancies and emphasizing the need for rigorous validation of such models used in policy-making.

Contribution

It provides a comparative analysis of wealth prediction models in Africa, revealing discrepancies and emphasizing validation for policy relevance.

Findings

Significant discrepancies between models' predictions.

Espín-Noboa et al. and Chi et al. have similar wealth distribution shapes.

Models' predictions vary notably even after data adjustments.

Abstract

Poverty map inference has become a critical focus of research, utilizing both traditional and modern techniques, ranging from regression models to convolutional neural networks applied to tabular data, satellite imagery, and networks. While much attention has been given to validating models during the training phase, the final predictions have received less scrutiny. In this study, we analyze the International Wealth Index (IWI) predicted by Lee and Braithwaite (2022) and Esp\'in-Noboa et al. (2023), alongside the Relative Wealth Index (RWI) inferred by Chi et al. (2022), across six Sub-Saharan African countries. Our analysis reveals trends and discrepancies in wealth predictions between these models. In particular, significant and unexpected discrepancies between the predictions of Lee and Braithwaite and Esp\'in-Noboa et al., even after accounting for differences in training data. In contrast, the shape of the wealth distributions predicted by Esp\'in-Noboa et al. and Chi et al. are more closely aligned, suggesting similar levels of skewness. These findings raise concerns about the validity of certain models and emphasize the importance of rigorous audits for wealth prediction algorithms used in policy-making. Continuous validation and refinement are essential to ensure the reliability of these models, particularly when they inform poverty alleviation strategies.