# Weighted Automata Extraction from Recurrent Neural Networks via Regression on State Spaces

## TL;DR

This paper introduces a novel method for extracting weighted finite automata from RNNs using regression techniques, enhancing the interpretability and analysis of neural network internal states.

## Contribution

It extends existing automaton extraction methods by incorporating regression for equivalence queries, enabling weighted automaton extraction from RNNs.

## Key findings

- High accuracy in automaton extraction
- Improved expressivity over previous DFA-based methods
- Efficient extraction process demonstrated

## Abstract

We present a method to extract a weighted finite automaton (WFA) from a recurrent neural network (RNN). Our algorithm is based on the WFA learning algorithm by Balle and Mohri, which is in turn an extension of Angluin's classic \lstar algorithm. Our technical novelty is in the use of \emph{regression} methods for the so-called equivalence queries, thus exploiting the internal state space of an RNN to prioritize counterexample candidates. This way we achieve a quantitative/weighted extension of the recent work by Weiss, Goldberg and Yahav that extracts DFAs. We experimentally evaluate the accuracy, expressivity and efficiency of the extracted WFAs.