Learning from Non-Binary Constituency Trees via Tensor Decomposition

TL;DR

This paper introduces a tensor decomposition-based method for processing non-binary constituency trees, avoiding binarisation and improving structural representation in NLP tasks.

Contribution

It proposes a novel tensor-based composition function with weight sharing for non-binary trees and integrates it into a Tree-LSTM model for enhanced NLP performance.

Findings

Effective handling of non-binary trees without binarisation

Parameter-efficient tensor decomposition approach

Improved NLP task performance with the new model

Abstract

Processing sentence constituency trees in binarised form is a common and popular approach in literature. However, constituency trees are non-binary by nature. The binarisation procedure changes deeply the structure, furthering constituents that instead are close. In this work, we introduce a new approach to deal with non-binary constituency trees which leverages tensor-based models. In particular, we show how a powerful composition function based on the canonical tensor decomposition can exploit such a rich structure. A key point of our approach is the weight sharing constraint imposed on the factor matrices, which allows limiting the number of model parameters. Finally, we introduce a Tree-LSTM model which takes advantage of this composition function and we experimentally assess its performance on different NLP tasks.