A Formal Language Approach to Explaining RNNs

TL;DR

This paper introduces LEXR, a formal framework using Linear Temporal Logic to generate interpretable explanations for RNN decisions, combining formal verification with machine learning techniques.

Contribution

It presents LEXR, a novel method that produces human-readable explanations for RNNs based on formal logic, with theoretical guarantees and empirical validation.

Findings

LEXR explanations are more accurate than recent automata-based methods.

LEXR provides explanations that are easier for humans to interpret.

The framework guarantees PAC-style correctness under certain conditions.

Abstract

This paper presents LEXR, a framework for explaining the decision making of recurrent neural networks (RNNs) using a formal description language called Linear Temporal Logic (LTL). LTL is the de facto standard for the specification of temporal properties in the context of formal verification and features many desirable properties that make the generated explanations easy for humans to interpret: it is a descriptive language, it has a variable-free syntax, and it can easily be translated into plain English. To generate explanations, LEXR follows the principle of counterexample-guided inductive synthesis and combines Valiant's probably approximately correct learning (PAC) with constraint solving. We prove that LEXR's explanations satisfy the PAC guarantee (provided the RNN can be described by LTL) and show empirically that these explanations are more accurate and easier-to-understand than the ones generated by recent algorithms that extract deterministic finite automata from RNNs.