Semantic Loss Application to Entity Relation Recognition

TL;DR

This paper introduces a novel end-to-end neural model for entity relation recognition that incorporates a unique loss function encoding logical constraints, improving performance and convergence speed.

Contribution

It presents a new loss function that encodes symbolic knowledge, enhancing neural models for joint entity relation extraction without feature engineering.

Findings

Models with the new loss outperform traditional models.

The proposed approach converges faster during training.

The methodology is applicable to broader language understanding tasks.

Abstract

Usually, entity relation recognition systems either use a pipe-lined model that treats the entity tagging and relation identification as separate tasks or a joint model that simultaneously identifies the relation and entities. This paper compares these two general approaches for the entity relation recognition. State-of-the-art entity relation recognition systems are built using deep recurrent neural networks which often does not capture the symbolic knowledge or the logical constraints in the problem. The main contribution of this paper is an end-to-end neural model for joint entity relation extraction which incorporates a novel loss function. This novel loss function encodes the constraint information in the problem to guide the model training effectively. We show that addition of this loss function to the existing typical loss functions has a positive impact over the performance of the models. This model is truly end-to-end, requires no feature engineering and easily extensible. Extensive experimentation has been conducted to evaluate the significance of capturing symbolic knowledge for natural language understanding. Models using this loss function are observed to be outperforming their counterparts and converging faster. Experimental results in this work suggest the use of this methodology for other language understanding applications.