Investigating the Influence of Legal Case Retrieval Systems on Users' Decision Process

TL;DR

This study investigates how legal case retrieval systems influence users' sentencing decisions, revealing that biased retrieval results can lead to harsher judgments, emphasizing the need for unbiased retrieval system design.

Contribution

First empirical study demonstrating the impact of retrieval result bias on legal users' decision-making processes.

Findings

Biased results lead to harsher sentencing judgments.

Legal users' decisions are significantly influenced by retrieval result distribution.

Highlighting the importance of unbiased legal retrieval systems.

Abstract

Given a specific query case, legal case retrieval systems aim to retrieve a set of case documents relevant to the case at hand. Previous studies on user behavior analysis have shown that information retrieval (IR) systems can significantly influence users' decisions by presenting results in varying orders and formats. However, whether such influence exists in legal case retrieval remains largely unknown. This study presents the first investigation into the influence of legal case retrieval systems on the decision-making process of legal users. We conducted an online user study involving more than ninety participants, and our findings suggest that the result distribution of legal case retrieval systems indeed affect users' judgements on the sentences in cases. Notably, when users are presented with biased results that involve harsher sentences, they tend to impose harsher sentences on the current case as well. This research highlights the importance of optimizing the unbiasedness of legal case retrieval systems.