Extensional Properties of Recurrent Neural Networks

TL;DR

This paper proves that determining whether a recurrent neural network has certain extensional properties, like robustness or clustering, is undecidable, highlighting fundamental limitations in analyzing RNNs.

Contribution

It establishes a Rice's theorem analogue for RNNs, showing that all nontrivial extensional properties are undecidable.

Findings

Any nontrivial extensional property of RNNs is undecidable.

The result applies broadly to properties like robustness and clustering.

Highlights fundamental limits in RNN analysis.

Abstract

A property of a recurrent neural network (RNN) is called \emph{extensional} if, loosely speaking, it is a property of the function computed by the RNN rather than a property of the RNN algorithm. Many properties of interest in RNNs are extensional, for example, robustness against small changes of input or good clustering of inputs. Given an RNN, it is natural to ask whether it has such a property. We give a negative answer to the general question about testing extensional properties of RNNs. Namely, we prove a version of Rice's theorem for RNNs: any nontrivial extensional property of RNNs is undecidable.