On the Fairness of 'Fake' Data in Legal AI

TL;DR

This paper critically examines the use of pre-processing bias correction methods in legal AI, highlighting legal and ethical challenges, and proposes alternative approaches to ensure fairness without compromising legal integrity.

Contribution

It provides a critique of pre-processing methods for bias correction in legal AI and offers recommendations for classifier modification or output correction to promote fairness.

Findings

Pre-processing can alter legal cases, affecting legal precedent.

Bias correction methods may introduce legal and ethical issues.

Alternative methods like classifier modification can improve fairness.

Abstract

The economics of smaller budgets and larger case numbers necessitates the use of AI in legal proceedings. We examine the concept of disparate impact and how biases in the training data lead to the search for fairer AI. This paper seeks to begin the discourse on what such an implementation would actually look like with a criticism of pre-processing methods in a legal context . We outline how pre-processing is used to correct biased data and then examine the legal implications of effectively changing cases in order to achieve a fairer outcome including the black box problem and the slow encroachment on legal precedent. Finally we present recommendations on how to avoid the pitfalls of pre-processed data with methods that either modify the classifier or correct the output in the final step.