The Role of Learning Algorithms in Collective Action

TL;DR

This paper investigates how the choice of learning algorithms, specifically distributionally robust optimization and stochastic gradient descent, influences the success of collective action in machine learning, emphasizing the importance of algorithm selection.

Contribution

It introduces the study of how learning algorithm choices affect collective success, supported by empirical and theoretical analysis, a perspective often overlooked in prior research.

Findings

Collective success varies significantly with the learning algorithm used.

Distributionally robust optimization improves worst group error.

SGD's inductive bias influences collective outcomes.

Abstract

Collective action in machine learning is the study of the control that a coordinated group can have over machine learning algorithms. While previous research has concentrated on assessing the impact of collectives against Bayes (sub-)optimal classifiers, this perspective is limited in that it does not account for the choice of learning algorithm. Since classifiers seldom behave like Bayes classifiers and are influenced by the choice of learning algorithms along with their inherent biases, in this work we initiate the study of how the choice of the learning algorithm plays a role in the success of a collective in practical settings. Specifically, we focus on distributionally robust optimization (DRO), popular for improving a worst group error, and on the ubiquitous stochastic gradient descent (SGD), due to its inductive bias for "simpler" functions. Our empirical results, supported by a theoretical foundation, show that the effective size and success of the collective are highly dependent on properties of the learning algorithm. This highlights the necessity of taking the learning algorithm into account when studying the impact of collective action in machine learning.