On the Possibility of Rewarding Structure Learning Agents: Mutual Information on Linguistic Random Sets

TL;DR

This paper explores a theoretical and empirical method to reward language structure learning agents by analyzing mutual information in semantic structures, potentially eliminating the need for pretrained analyzers.

Contribution

It introduces a novel approach to reward design for language agents using mutual information on random sets, without relying on pretrained structural analyzers.

Findings

Mutual information distinguishes semantic structures from random sets.

Empirical evidence supports rewarding agents based on mutual information.

Potential to reward structure learning without pretrained analyzers.

Abstract

We present a first attempt to elucidate a theoretical and empirical approach to design the reward provided by a natural language environment to some structure learning agent. To this end, we revisit the Information Theory of unsupervised induction of phrase-structure grammars to characterize the behavior of simulated actions modeled as set-valued random variables (random sets of linguistic samples) constituting semantic structures. Our results showed empirical evidence of that simulated semantic structures (Open Information Extraction triplets) can be distinguished from randomly constructed ones by observing the Mutual Information among their constituents. This suggests the possibility of rewarding structure learning agents without using pretrained structural analyzers (oracle actors/experts).