Convolutional Lie Operator for Sentence Classification

TL;DR

This paper introduces Lie Convolutions into sentence classifiers, capturing complex language transformations and outperforming traditional CNN models in empirical tests.

Contribution

It presents a novel integration of Lie group operations into convolutional sentence classifiers, enhancing their ability to model complex language transformations.

Findings

Lie-based models outperform traditional CNN classifiers in accuracy.

Lie Convolutions capture non-Euclidean symmetries in language.

Empirical results show significant improvements in sentence classification tasks.

Abstract

Traditional Convolutional Neural Networks have been successful in capturing local, position-invariant features in text, but their capacity to model complex transformation within language can be further explored. In this work, we explore a novel approach by integrating Lie Convolutions into Convolutional-based sentence classifiers, inspired by the ability of Lie group operations to capture complex, non-Euclidean symmetries. Our proposed models SCLie and DPCLie empirically outperform traditional Convolutional-based sentence classifiers, suggesting that Lie-based models relatively improve the accuracy by capturing transformations not commonly associated with language. Our findings motivate more exploration of new paradigms in language modeling.