AI and the Law: Evaluating ChatGPT's Performance in Legal Classification

TL;DR

This study evaluates ChatGPT's ability to classify legal cases under the Polish Penal Code, demonstrating high accuracy and appropriate legal reasoning, indicating its potential as a legal analysis tool.

Contribution

It is the first to assess ChatGPT's performance in Polish legal classification, showing its effectiveness in evidence analysis and legal reasoning.

Findings

Excellent binary classification accuracy for legal cases

Legal basis and content provided by ChatGPT were appropriate

Potential for ChatGPT to assist in legal evidence analysis

Abstract

The use of ChatGPT to analyze and classify evidence in criminal proceedings has been a topic of ongoing discussion. However, to the best of our knowledge, this issue has not been studied in the context of the Polish language. This study addresses this research gap by evaluating the effectiveness of ChatGPT in classifying legal cases under the Polish Penal Code. The results show excellent binary classification accuracy, with all positive and negative cases correctly categorized. In addition, a qualitative evaluation confirms that the legal basis provided for each case, along with the relevant legal content, was appropriate. The results obtained suggest that ChatGPT can effectively analyze and classify evidence while applying the appropriate legal rules. In conclusion, ChatGPT has the potential to assist interested parties in the analysis of evidence and serve as a valuable legal resource for individuals with less experience or knowledge in this area.