INACIA: Integrating Large Language Models in Brazilian Audit Courts: Opportunities and Challenges

TL;DR

This paper presents INACIA, a system integrating Large Language Models into Brazilian Court operations, automating case analysis and decision support, demonstrating promising results with high correlation to human judgments.

Contribution

Introduces INACIA, a novel AI system that automates legal case analysis in Brazilian courts, with a new evaluation methodology correlating system performance with human judgment.

Findings

INACIA effectively extracts relevant case information.

The system's legal plausibility evaluation aligns closely with human assessments.

Demonstrates potential for AI to assist in complex legal decision-making.

Abstract

This paper introduces INACIA (Instru\c{c}\~ao Assistida com Intelig\^encia Artificial), a groundbreaking system designed to integrate Large Language Models (LLMs) into the operational framework of Brazilian Federal Court of Accounts (TCU). The system automates various stages of case analysis, including basic information extraction, admissibility examination, Periculum in mora and Fumus boni iuris analyses, and recommendations generation. Through a series of experiments, we demonstrate INACIA's potential in extracting relevant information from case documents, evaluating its legal plausibility, and formulating propositions for judicial decision-making. Utilizing a validation dataset alongside LLMs, our evaluation methodology presents a novel approach to assessing system performance, correlating highly with human judgment. These results underscore INACIA's potential in complex legal task handling while also acknowledging the current limitations. This study discusses possible improvements and the broader implications of applying AI in legal contexts, suggesting that INACIA represents a significant step towards integrating AI in legal systems globally, albeit with cautious optimism grounded in the empirical findings.