Co-Matching: Towards Human-Machine Collaborative Legal Case Matching

TL;DR

This paper introduces Co-Matching, a collaborative framework that combines human legal practitioners' tacit knowledge with AI to improve legal case matching accuracy and collaboration effectiveness.

Contribution

It proposes a novel human-machine collaborative matching framework and a method called ProtoEM for estimating decision uncertainty, advancing legal case matching techniques.

Findings

Co-Matching outperforms existing methods with +5.51% and +8.71% improvements.

It enhances human-machine collaboration effectiveness.

Experimental results validate the approach's superiority.

Abstract

Recent efforts have aimed to improve AI machines in legal case matching by integrating legal domain knowledge. However, successful legal case matching requires the tacit knowledge of legal practitioners, which is difficult to verbalize and encode into machines. This emphasizes the crucial role of involving legal practitioners in high-stakes legal case matching. To address this, we propose a collaborative matching framework called Co-Matching, which encourages both the machine and the legal practitioner to participate in the matching process, integrating tacit knowledge. Unlike existing methods that rely solely on the machine, Co-Matching allows both the legal practitioner and the machine to determine key sentences and then combine them probabilistically. Co-Matching introduces a method called ProtoEM to estimate human decision uncertainty, facilitating the probabilistic combination. Experimental results demonstrate that Co-Matching consistently outperforms existing legal case matching methods, delivering significant performance improvements over human- and machine-based matching in isolation (on average, +5.51% and +8.71%, respectively). Further analysis shows that Co-Matching also ensures better human-machine collaboration effectiveness. Our study represents a pioneering effort in human-machine collaboration for the matching task, marking a milestone for future collaborative matching studies.