Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Models

TL;DR

This study evaluates GPT-4's ability to automatically extract key legal information from UK Employment Tribunal judgments, demonstrating high accuracy and potential for legal research and dispute outcome prediction.

Contribution

It is the first comprehensive evaluation of GPT-4 for extracting detailed legal case information from tribunal judgments, with a focus on practical applications.

Findings

GPT-4 achieves high accuracy in extracting legal case details.

The extracted data can be used to predict case outcomes.

Demonstrates the potential of LLMs in legal information processing.

Abstract

Court transcripts and judgments are rich repositories of legal knowledge, detailing the intricacies of cases and the rationale behind judicial decisions. The extraction of key information from these documents provides a concise overview of a case, crucial for both legal experts and the public. With the advent of large language models (LLMs), automatic information extraction has become increasingly feasible and efficient. This paper presents a comprehensive study on the application of GPT-4, a large language model, for automatic information extraction from UK Employment Tribunal (UKET) cases. We meticulously evaluated GPT-4's performance in extracting critical information with a manual verification process to ensure the accuracy and relevance of the extracted data. Our research is structured around two primary extraction tasks: the first involves a general extraction of eight key aspects that hold significance for both legal specialists and the general public, including the facts of the case, the claims made, references to legal statutes, references to precedents, general case outcomes and corresponding labels, detailed order and remedies and reasons for the decision. The second task is more focused, aimed at analysing three of those extracted features, namely facts, claims and outcomes, in order to facilitate the development of a tool capable of predicting the outcome of employment law disputes. Through our analysis, we demonstrate that LLMs like GPT-4 can obtain high accuracy in legal information extraction, highlighting the potential of LLMs in revolutionising the way legal information is processed and utilised, offering significant implications for legal research and practice.