Conditional probing: measuring usable information beyond a baseline

TL;DR

This paper introduces conditional probing, a method to measure the information in neural representations beyond what is captured by baseline embeddings, revealing deeper insights into neural network layers.

Contribution

It extends $ ext{V}$-information theory to develop conditional probing, enabling detection of information not explained by baselines in neural representations.

Findings

Part-of-speech information is accessible at deeper layers after conditioning on word embeddings.

Conditional probing reveals more nuanced information in neural networks than traditional baseline comparisons.

The method uncovers hidden layers of information beyond baseline explanations.

Abstract

Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings. Instead of using baselines as a point of comparison, we're interested in measuring information that is contained in the representation but not in the baseline. For example, current methods can detect when a representation is more useful than the word identity (a baseline) for predicting part-of-speech; however, they cannot detect when the representation is predictive of just the aspects of part-of-speech not explainable by the word identity. In this work, we extend a theory of usable information called $\mathcal{V}$-information and propose conditional probing, which explicitly conditions on the information in the baseline. In a case study, we find that after conditioning on non-contextual word embeddings, properties like part-of-speech are accessible at deeper layers of a network than previously thought.