An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks

TL;DR

This paper introduces an automata-theoretic method for synthesizing binarized neural networks (BNNs) that satisfy specific properties, enhancing fairness and robustness while controlling network parameters.

Contribution

It proposes a novel automata-based synthesis framework for BNNs using a new temporal logic, BLTL, and SMT solvers, addressing interpretability and property verification.

Findings

Improves fairness and robustness of BNNs

Maintains high accuracy in synthesized networks

Provides a method to determine hyper-parameters before training

Abstract

Deep neural networks, (DNNs, a.k.a. NNs), have been widely used in various tasks and have been proven to be successful. However, the accompanied expensive computing and storage costs make the deployments in resource-constrained devices a significant concern. To solve this issue, quantization has emerged as an effective way to reduce the costs of DNNs with little accuracy degradation by quantizing floating-point numbers to low-width fixed-point representations. Quantized neural networks (QNNs) have been developed, with binarized neural networks (BNNs) restricted to binary values as a special case. Another concern about neural networks is their vulnerability and lack of interpretability. Despite the active research on trustworthy of DNNs, few approaches have been proposed to QNNs. To this end, this paper presents an automata-theoretic approach to synthesizing BNNs that meet designated properties. More specifically, we define a temporal logic, called BLTL, as the specification language. We show that each BLTL formula can be transformed into an automaton on finite words. To deal with the state-explosion problem, we provide a tableau-based approach in real implementation. For the synthesis procedure, we utilize SMT solvers to detect the existence of a model (i.e., a BNN) in the construction process. Notably, synthesis provides a way to determine the hyper-parameters of the network before training.Moreover, we experimentally evaluate our approach and demonstrate its effectiveness in improving the individual fairness and local robustness of BNNs while maintaining accuracy to a great extent.