# A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations

## TL;DR

This paper introduces a generative model that disentangles syntax and semantics in sentence representations using multiple training losses and neural network modules, improving interpretability and task performance.

## Contribution

It presents a novel multi-task generative model that enhances disentanglement of syntax and semantics in sentence embeddings, with comprehensive evaluation.

## Key findings

- Better disentanglement achieved with multi-loss training
- Recurrent modules improve syntactic and semantic representations
- Model outperforms pretrained embeddings on similarity tasks

## Abstract

We propose a generative model for a sentence that uses two latent variables, with one intended to represent the syntax of the sentence and the other to represent its semantics. We show we can achieve better disentanglement between semantic and syntactic representations by training with multiple losses, including losses that exploit aligned paraphrastic sentences and word-order information. We also investigate the effect of moving from bag-of-words to recurrent neural network modules. We evaluate our models as well as several popular pretrained embeddings on standard semantic similarity tasks and novel syntactic similarity tasks. Empirically, we find that the model with the best performing syntactic and semantic representations also gives rise to the most disentangled representations.