Deep Binary Reinforcement Learning for Scalable Verification

TL;DR

This paper introduces a reinforcement learning approach using binarized neural networks to enhance the scalability of verifying safety properties in deep RL policies, demonstrated on Atari games.

Contribution

It presents a novel RL algorithm specifically designed for binarized neural networks, improving the verification process for large-scale deep RL policies.

Findings

Successfully trained BNNs on Atari environments

Verified robustness properties of the trained BNN policies

Enhanced scalability of neural network verification in RL

Abstract

The use of neural networks as function approximators has enabled many advances in reinforcement learning (RL). The generalization power of neural networks combined with advances in RL algorithms has reignited the field of artificial intelligence. Despite their power, neural networks are considered black boxes, and their use in safety-critical settings remains a challenge. Recently, neural network verification has emerged as a way to certify safety properties of networks. Verification is a hard problem, and it is difficult to scale to large networks such as the ones used in deep reinforcement learning. We provide an approach to train RL policies that are more easily verifiable. We use binarized neural networks (BNNs), a type of network with mostly binary parameters. We present an RL algorithm tailored specifically for BNNs. After training BNNs for the Atari environments, we verify robustness properties.