Verifying Properties of Tsetlin Machines

TL;DR

This paper introduces a formal verification method for Tsetlin Machines by encoding them into propositional logic and using SAT solvers to check properties like robustness, similarity, and equivalence, demonstrated on MNIST and IMDB datasets.

Contribution

It provides the first exact encoding of TsMs into propositional logic and applies formal verification techniques to assess their robustness and similarity.

Findings

Verified adversarial robustness of TsMs on MNIST.

Established formal notions of similarity and equivalence for TsMs.

Compared TsM robustness with Binarized Neural Networks.

Abstract

Tsetlin Machines (TsMs) are a promising and interpretable machine learning method which can be applied for various classification tasks. We present an exact encoding of TsMs into propositional logic and formally verify properties of TsMs using a SAT solver. In particular, we introduce in this work a notion of similarity of machine learning models and apply our notion to check for similarity of TsMs. We also consider notions of robustness and equivalence from the literature and adapt them for TsMs. Then, we show the correctness of our encoding and provide results for the properties: adversarial robustness, equivalence, and similarity of TsMs. In our experiments, we employ the MNIST and IMDB datasets for (respectively) image and sentiment classification. We discuss the results for verifying robustness obtained with TsMs with those in the literature obtained with Binarized Neural Networks on MNIST.