An Empirical Study on Transfer Learning for Privilege Review

TL;DR

This study evaluates the effectiveness of transfer learning, especially BERT, in identifying privileged legal documents, demonstrating improved performance over traditional methods across multiple datasets.

Contribution

It provides empirical evidence that transfer learning with BERT enhances privilege document classification in legal review, outperforming standard logistic regression models.

Findings

BERT outperforms logistic regression in privilege classification.

Transfer learning achieves decent results on similar domain datasets.

Models perform well with limited new training data.

Abstract

Protecting privileged communications and data from inadvertent disclosure is a paramount task in the US legal practice. Traditionally counsels rely on keyword searching and manual review to identify privileged documents in cases. As data volumes increase, this approach becomes less and less defensible in costs. Machine learning methods have been used in identifying privilege documents. Given the generalizable nature of privilege in legal cases, we hypothesize that transfer learning can capitalize knowledge learned from existing labeled data to identify privilege documents without requiring labeling new training data. In this paper, we study both traditional machine learning models and deep learning models based on BERT for privilege document classification tasks in legal document review, and we examine the effectiveness of transfer learning in privilege model on three real world datasets with privilege labels. Our results show that BERT model outperforms the industry standard logistic regression algorithm and transfer learning models can achieve decent performance on datasets in same or close domains.