Is neural semantic parsing good at ellipsis resolution, or isn't it?

TL;DR

Neural semantic parsers excel generally but struggle with context-sensitive phenomena like verb phrase ellipsis, especially in complex linguistic contexts, though data augmentation can improve their performance.

Contribution

This study evaluates neural semantic parsers on ellipsis resolution, revealing their limitations and demonstrating that data augmentation can enhance their handling of complex linguistic phenomena.

Findings

Parsers perform well on standard tests but poorly on ellipsis cases.

Data augmentation improves ellipsis resolution accuracy.

Complex linguistic contexts cause most parsing errors.

Abstract

Neural semantic parsers have shown good overall performance for a variety of linguistic phenomena, reaching semantic matching scores of more than 90%. But how do such parsers perform on strongly context-sensitive phenomena, where large pieces of semantic information need to be duplicated to form a meaningful semantic representation? A case in point is English verb phrase ellipsis, a construct where entire verb phrases can be abbreviated by a single auxiliary verb. Are the otherwise known as powerful semantic parsers able to deal with ellipsis or aren't they? We constructed a corpus of 120 cases of ellipsis with their fully resolved meaning representation and used this as a challenge set for a large battery of neural semantic parsers. Although these parsers performed very well on the standard test set, they failed in the instances with ellipsis. Data augmentation helped improve the parsing results. The reason for the difficulty of parsing elided phrases is not that copying semantic material is hard, but that usually occur in linguistically complicated contexts causing most of the parsing errors.