Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples

TL;DR

This paper introduces a novel method that employs exact learning and abstraction to extract deterministic finite automata from trained RNNs, enabling better understanding of their state dynamics.

Contribution

The paper presents a new algorithm combining Angluin's L* algorithm with RNNs as oracles to accurately extract automata from complex neural networks.

Findings

Efficient extraction of automata from large RNNs.

High accuracy in modeling RNN state dynamics.

Applicable to various trained RNN architectures.

Abstract

We present a novel algorithm that uses exact learning and abstraction to extract a deterministic finite automaton describing the state dynamics of a given trained RNN. We do this using Angluin's L* algorithm as a learner and the trained RNN as an oracle. Our technique efficiently extracts accurate automata from trained RNNs, even when the state vectors are large and require fine differentiation.