Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems

TL;DR

This paper examines the accuracy-efficiency trade-offs in distributed machine learning systems, emphasizing policy implications, accountability, and governance to ensure safety and transparency in high-stakes applications.

Contribution

It analyzes policy gaps and proposes accountability mechanisms for managing accuracy-efficiency trade-offs in distributed ML systems, especially in safety-critical contexts.

Findings

Identifies policy gaps in US risk assessment standards for distributed ML

Highlights the importance of accountability for safety-critical systems

Proposes actions to improve governance and transparency

Abstract

Trade-offs between accuracy and efficiency pervade law, public health, and other non-computing domains, which have developed policies to guide how to balance the two in conditions of uncertainty. While computer science also commonly studies accuracy-efficiency trade-offs, their policy implications remain poorly examined. Drawing on risk assessment practices in the US, we argue that, since examining these trade-offs has been useful for guiding governance in other domains, we need to similarly reckon with these trade-offs in governing computer systems. We focus our analysis on distributed machine learning systems. Understanding the policy implications in this area is particularly urgent because such systems, which include autonomous vehicles, tend to be high-stakes and safety-critical. We 1) describe how the trade-off takes shape for these systems, 2) highlight gaps between existing US risk assessment standards and what these systems require to be properly assessed, and 3) make specific calls to action to facilitate accountability when hypothetical risks concerning the accuracy-efficiency trade-off become realized as accidents in the real world. We close by discussing how such accountability mechanisms encourage more just, transparent governance aligned with public values.