Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development

TL;DR

This paper explores how resource constraints in smaller organizations impact responsible machine learning development, highlighting unique challenges and ethical considerations often overlooked in big tech-focused studies.

Contribution

It provides a qualitative analysis of interviews with practitioners from underrepresented organizations, revealing specific tensions caused by resource limitations.

Findings

Resource constraints lead to privacy and ubiquity tensions.

Limited resources affect performance optimization.

Access issues contribute to monopolization concerns.

Abstract

Practitioners from diverse occupations and backgrounds are increasingly using machine learning (ML) methods. Nonetheless, studies on ML Practitioners typically draw populations from Big Tech and academia, as researchers have easier access to these communities. Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example, practitioners working at startups, at non-tech companies, and in the public sector. These practitioners share many of the same ML development difficulties and ethical conundrums as their Big Tech counterparts; however, their experiences are subject to additional under-studied challenges stemming from deploying ML with limited resources, increased existential risk, and absent access to in-house research teams. We contribute a qualitative analysis of 17 interviews with stakeholders from organizations which are less represented in prior studies. We uncover a number of tensions which are introduced or exacerbated by these organizations' resource constraints -- tensions between privacy and ubiquity, resource management and performance optimization, and access and monopolization. Increased academic focus on these practitioners can facilitate a more holistic understanding of ML limitations, and so is useful for prescribing a research agenda to facilitate responsible ML development for all.