Explainability by design: an experimental analysis of the legal coding process

TL;DR

This paper presents a methodology for legal coding using Deontic Defeasible Logic, analyzing human efforts and proposing a way to predict coding time based on various factors, enhancing explainability in legal AI systems.

Contribution

It introduces a novel methodology for legal coding with Deontic Defeasible Logic and provides an experimental analysis of human efforts and a predictive technique for coding time.

Findings

Human effort varies with legal domain knowledge and text complexity.

The Houdini technology effectively supports reasoning in Deontic Defeasible Logic.

A model predicts coding time based on factors like text length and legal reference depth.

Abstract

Behind a set of rules in Deontic Defeasible Logic, there is a mapping process of normative background fragments. This process goes from text to rules and implicitly encompasses an explanation of the coded fragments. In this paper we deliver a methodology for \textit{legal coding} that starts with a fragment and goes onto a set of Deontic Defeasible Logic rules, involving a set of \textit{scenarios} to test the correctness of the coded fragments. The methodology is illustrated by the coding process of an example text. We then show the results of a series of experiments conducted with humans encoding a variety of normative backgrounds and corresponding cases in which we have measured the efforts made in the coding process, as related to some measurable features. To process these examples, a recently developed technology, Houdini, that allows reasoning in Deontic Defeasible Logic, has been employed. Finally we provide a technique to forecast time required in coding, that depends on factors such as knowledge of the legal domain, knowledge of the coding processes, length of the text, and a measure of \textit{depth} that refers to the length of the paths of legal references.