Two Birds with One Stone: Investigating Invertible Neural Networks for Inverse Problems in Morphology

TL;DR

This paper explores the use of Invertible Neural Networks to simultaneously solve morphological analysis and generation tasks, demonstrating competitive performance and efficiency in handling inverse problems in NLP.

Contribution

It introduces the application of INNs to morphological inverse problems, enabling a single model to perform analysis and generation without performance loss.

Findings

INNs can effectively handle morphological inverse problems.

The model performs well in both analysis and generation tasks.

No significant performance drop observed in either direction.

Abstract

Most problems in natural language processing can be approximated as inverse problems such as analysis and generation at variety of levels from morphological (e.g., cat+Plural <-> cats) to semantic (e.g., (call + 1 2) <-> "Calculate one plus two."). Although the tasks in both directions are closely related, general approach in the field has been to design separate models specific for each task. However, having one shared model for both tasks, would help the researchers exploit the common knowledge among these problems with reduced time and memory requirements. We investigate a specific class of neural networks, called Invertible Neural Networks (INNs) (Ardizzone et al. 2019) that enable simultaneous optimization in both directions, hence allow addressing of inverse problems via a single model. In this study, we investigate INNs on morphological problems casted as inverse problems. We apply INNs to various morphological tasks with varying ambiguity and show that they provide competitive performance in both directions. We show that they are able to recover the morphological input parameters, i.e., predicting the lemma (e.g., cat) or the morphological tags (e.g., Plural) when run in the reverse direction, without any significant performance drop in the forward direction, i.e., predicting the surface form (e.g., cats).