On Controlled DeEntanglement for Natural Language Processing

TL;DR

This paper proposes a framework for controlled de-entanglement in AI models, using information theory to improve scalability, flexibility, and interpretability in medium-stake scenarios like Visual Dialog.

Contribution

It introduces a novel approach employing stochasticity for controlled de-entanglement, supported by mathematical analysis and initial experimental results.

Findings

Stochasticity enables controlled de-entanglement of relevant factors.

Initial experiments demonstrate the framework's efficacy.

Roadmap for future experiments on scalability and interpretability.

Abstract

Latest addition to the toolbox of human species is Artificial Intelligence(AI). Thus far, AI has made significant progress in low stake low risk scenarios such as playing Go and we are currently in a transition toward medium stake scenarios such as Visual Dialog. In my thesis, I argue that we need to incorporate controlled de-entanglement as first class object to succeed in this transition. I present mathematical analysis from information theory to show that employing stochasticity leads to controlled de-entanglement of relevant factors of variation at various levels. Based on this, I highlight results from initial experiments that depict efficacy of the proposed framework. I conclude this writeup by a roadmap of experiments that show the applicability of this framework to scalability, flexibility and interpretibility.