A Case for Rejection in Low Resource ML Deployment

TL;DR

This paper discusses the challenge of deploying reliable ML systems in low-resource settings and proposes a simple rejection-based method as a baseline to address data scarcity issues.

Contribution

It highlights the limitations of existing rejection methods in low-resource scenarios and introduces a straightforward baseline solution.

Findings

Rejection can improve model reliability in resource-limited environments

The proposed baseline offers a simple approach for sample rejection in low-resource ML

Initial results suggest potential benefits of rejection methods in deployment scenarios

Abstract

Building reliable AI decision support systems requires a robust set of data on which to train models; both with respect to quantity and diversity. Obtaining such datasets can be difficult in resource limited settings, or for applications in early stages of deployment. Sample rejection is one way to work around this challenge, however much of the existing work in this area is ill-suited for such scenarios. This paper substantiates that position and proposes a simple solution as a proof of concept baseline.