PDFA Distillation via String Probability Queries

TL;DR

This paper introduces a novel algorithm for distilling probabilistic deterministic finite automata (PDFA) from neural networks using string probability queries, enhancing explainability in language models.

Contribution

It presents a new distillation method based on the L# algorithm that infers conditional probabilities from string occurrence probabilities.

Findings

Effective distillation of PDFA from neural networks demonstrated on public datasets.

The algorithm accurately captures probabilistic language structures.

Improves interpretability of neural language models.

Abstract

Probabilistic deterministic finite automata (PDFA) are discrete event systems modeling conditional probabilities over languages: Given an already seen sequence of tokens they return the probability of tokens of interest to appear next. These types of models have gained interest in the domain of explainable machine learning, where they are used as surrogate models for neural networks trained as language models. In this work we present an algorithm to distill PDFA from neural networks. Our algorithm is a derivative of the L# algorithm and capable of learning PDFA from a new type of query, in which the algorithm infers conditional probabilities from the probability of the queried string to occur. We show its effectiveness on a recent public dataset by distilling PDFA from a set of trained neural networks.