Goodness-of-fit and utility estimation: what's possible and what's not

TL;DR

This paper investigates the limitations of goodness-of-fit indices in utility-maximization models, showing that no perfect, continuous index exists for well-behaved utility functions, but welfare comparisons remain feasible.

Contribution

It demonstrates the fundamental impossibility of a perfect goodness-of-fit index for continuous, increasing utility functions and proposes alternative methods for welfare comparison.

Findings

No continuous, accurate goodness-of-fit index exists for well-behaved utility functions.

Standard loss-based indices are generally inaccurate for perfect fit.

Welfare comparisons can be reliably made through robust preference relations.

Abstract

A goodness-of-fit index measures the consistency of consumption data with a given model of utility-maximization. We show that for the class of well-behaved (i.e., continuous and increasing) utility functions there is no goodness-of-fit index that is continuous and accurate, where the latter means that a perfect score is obtained if and only if a dataset can be rationalized by a well-behaved utility function. While many standard goodness-of-fit indices are inaccurate we show that these indices are (in a sense we make precise) essentially accurate. Goodness-of-fit indices are typically generated by loss functions and we find that standard loss functions usually do not yield a best-fitting utility function when they are minimized. Nonetheless, welfare comparisons can be made by working out a robust preference relation from the data.