AMR Quality Rating with a Lightweight CNN

TL;DR

This paper introduces a lightweight CNN model that accurately assesses the quality of AMR parses without gold data, by transforming AMRs into images and mimicking human judgment, improving accuracy and efficiency.

Contribution

The novel approach of converting AMRs into images enables a simple CNN to effectively rate AMR quality without gold annotations, outperforming strong baselines.

Findings

Outperforms baseline quality rating methods

More accurate in multiple quality dimensions

Reduces energy consumption

Abstract

Structured semantic sentence representations such as Abstract Meaning Representations (AMRs) are potentially useful in various NLP tasks. However, the quality of automatic parses can vary greatly and jeopardizes their usefulness. This can be mitigated by models that can accurately rate AMR quality in the absence of costly gold data, allowing us to inform downstream systems about an incorporated parse's trustworthiness or select among different candidate parses. In this work, we propose to transfer the AMR graph to the domain of images. This allows us to create a simple convolutional neural network (CNN) that imitates a human judge tasked with rating graph quality. Our experiments show that the method can rate quality more accurately than strong baselines, in several quality dimensions. Moreover, the method proves to be efficient and reduces the incurred energy consumption.