Evaluation Gaps in Machine Learning Practice

TL;DR

This paper investigates the limited scope of current ML evaluation practices, highlighting the neglect of important contextual and normative factors, and advocates for more comprehensive, context-aware evaluation methods to ensure responsible ML deployment.

Contribution

It empirically analyzes evaluation practices in top ML conferences, revealing implicit normative assumptions and proposing the need for more contextualized evaluation methodologies.

Findings

Focus on narrow evaluation metrics in CV and NLP

Neglect of contextual and normative properties in evaluations

Implicit commitments like consequentialism and quantifiability influence evaluation choices

Abstract

Forming a reliable judgement of a machine learning (ML) model's appropriateness for an application ecosystem is critical for its responsible use, and requires considering a broad range of factors including harms, benefits, and responsibilities. In practice, however, evaluations of ML models frequently focus on only a narrow range of decontextualized predictive behaviours. We examine the evaluation gaps between the idealized breadth of evaluation concerns and the observed narrow focus of actual evaluations. Through an empirical study of papers from recent high-profile conferences in the Computer Vision and Natural Language Processing communities, we demonstrate a general focus on a handful of evaluation methods. By considering the metrics and test data distributions used in these methods, we draw attention to which properties of models are centered in the field, revealing the properties that are frequently neglected or sidelined during evaluation. By studying these properties, we demonstrate the machine learning discipline's implicit assumption of a range of commitments which have normative impacts; these include commitments to consequentialism, abstractability from context, the quantifiability of impacts, the limited role of model inputs in evaluation, and the equivalence of different failure modes. Shedding light on these assumptions enables us to question their appropriateness for ML system contexts, pointing the way towards more contextualized evaluation methodologies for robustly examining the trustworthiness of ML models