TAI Scan Tool: A RAG-Based Tool With Minimalistic Input for Trustworthy AI Self-Assessment

TL;DR

The TAI Scan Tool is a RAG-based self-assessment system designed to evaluate AI systems' compliance with the AI Act, providing risk insights and relevant legal articles with minimal input.

Contribution

This paper presents a novel RAG-based tool for trustworthy AI self-assessment that supports legal compliance with minimal input and offers interpretability of risk evaluation.

Findings

Accurately predicts AI system risk levels in use-case scenarios.

Retrieves relevant legal articles to support compliance.

Demonstrates effective reasoning aligned with high-risk AI settings.

Abstract

This paper introduces the TAI Scan Tool, a RAG-based TAI self-assessment tool with minimalistic input. The current version of the tool supports the legal TAI assessment, with a particular emphasis on facilitating compliance with the AI Act. It involves a two-step approach with a pre-screening and an assessment phase. The assessment output of the system includes insight regarding the risk-level of the AI system according to the AI Act, while at the same time retrieving relevant articles to aid with compliance and notify on their obligations. Our qualitative evaluation using use-case scenarios yields promising results, correctly predicting risk levels while retrieving relevant articles across three distinct semantic groups. Furthermore, interpretation of results shows that the tool's reasoning relies on comparison with the setting of high-risk systems, a behaviour attributed to their deployment requiring careful consideration, and therefore frequently presented within the AI Act.