Learning Disentangled Representations of Negation and Uncertainty

TL;DR

This paper proposes a method using a Variational Autoencoder to explicitly disentangle representations of negation, uncertainty, and content in natural language processing, improving interpretability and modeling accuracy.

Contribution

It introduces a novel approach to disentangle negation, uncertainty, and content representations, with simple supervision and auxiliary objectives enhancing the separation.

Findings

Supervised latent representations achieve good disentanglement.

Adversarial learning and mutual information minimization improve disentanglement further.

The method demonstrates effective separation of negation, uncertainty, and content in NLP models.

Abstract

Negation and uncertainty modeling are long-standing tasks in natural language processing. Linguistic theory postulates that expressions of negation and uncertainty are semantically independent from each other and the content they modify. However, previous works on representation learning do not explicitly model this independence. We therefore attempt to disentangle the representations of negation, uncertainty, and content using a Variational Autoencoder. We find that simply supervising the latent representations results in good disentanglement, but auxiliary objectives based on adversarial learning and mutual information minimization can provide additional disentanglement gains.