Non-Determinism and the Lawlessness of Machine Learning Code

TL;DR

This paper explores the role of non-determinism in machine learning, emphasizing its implications for law by viewing ML outputs as distributions over possible outcomes, thus challenging the assumption of code determinism in cyberlaw.

Contribution

It clarifies the distinction between stochasticity and non-determinism in ML, proposing a distributional perspective that enhances legal analysis beyond individual outcomes.

Findings

Non-determinism affects ML outcomes as distributions, not fixed results.

ML code is inherently non-deterministic, challenging the 'code as law' paradigm.

Legal frameworks need to adapt to the distributional nature of ML outputs.

Abstract

Legal literature on machine learning (ML) tends to focus on harms, and thus tends to reason about individual model outcomes and summary error rates. This focus has masked important aspects of ML that are rooted in its reliance on randomness -- namely, stochasticity and non-determinism. While some recent work has begun to reason about the relationship between stochasticity and arbitrariness in legal contexts, the role of non-determinism more broadly remains unexamined. In this paper, we clarify the overlap and differences between these two concepts, and show that the effects of non-determinism, and consequently its implications for the law, become clearer from the perspective of reasoning about ML outputs as distributions over possible outcomes. This distributional viewpoint accounts for randomness by emphasizing the possible outcomes of ML. Importantly, this type of reasoning is not exclusive with current legal reasoning; it complements (and in fact can strengthen) analyses concerning individual, concrete outcomes for specific automated decisions. By illuminating the important role of non-determinism, we demonstrate that ML code falls outside of the cyberlaw frame of treating ``code as law,'' as this frame assumes that code is deterministic. We conclude with a brief discussion of what work ML can do to constrain the potentially harm-inducing effects of non-determinism, and we indicate where the law must do work to bridge the gap between its current individual-outcome focus and the distributional approach that we recommend.