Multitasking Inhibits Semantic Drift
Athul Paul Jacob, Mike Lewis, Jacob Andreas

TL;DR
This paper investigates how multitask training can prevent semantic drift in latent language policies used by agents in complex tasks, improving their communication consistency and efficiency.
Contribution
It provides theoretical proof and empirical evidence that multitask training eliminates semantic drift in latent language policies.
Findings
Multitask training eliminates semantic drift in signaling games.
Multitask training reduces semantic drift in neural language policies.
Multitask training improves sample efficiency in complex strategy games.
Abstract
When intelligent agents communicate to accomplish shared goals, how do these goals shape the agents' language? We study the dynamics of learning in latent language policies (LLPs), in which instructor agents generate natural-language subgoal descriptions and executor agents map these descriptions to low-level actions. LLPs can solve challenging long-horizon reinforcement learning problems and provide a rich model for studying task-oriented language use. But previous work has found that LLP training is prone to semantic drift (use of messages in ways inconsistent with their original natural language meanings). Here, we demonstrate theoretically and empirically that multitask training is an effective counter to this problem: we prove that multitask training eliminates semantic drift in a well-studied family of signaling games, and show that multitask training of neural LLPs in a complex…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Reinforcement Learning in Robotics · Natural Language Processing Techniques
