Evaluating Induced CCG Parsers on Grounded Semantic Parsing
Yonatan Bisk, Siva Reddy, John Blitzer, Julia Hockenmaier, Mark, Steedman
TL;DR
This paper evaluates four different syntactic CCG parsers on a new semantic slot-filling dataset to understand how much syntactic supervision is needed for effective semantic analysis.
Contribution
It introduces SPADES, a new large-scale semantic parsing dataset, and compares the performance of various CCG parsers in a task-based evaluation setting.
Findings
Unsupervised grammar induction systems have varying strengths in semantic tasks.
Grounded semantic parsing benefits from specific syntactic features.
The dataset enables comprehensive evaluation of semantic parsing models.
Abstract
We compare the effectiveness of four different syntactic CCG parsers for a semantic slot-filling task to explore how much syntactic supervision is required for downstream semantic analysis. This extrinsic, task-based evaluation provides a unique window to explore the strengths and weaknesses of semantics captured by unsupervised grammar induction systems. We release a new Freebase semantic parsing dataset called SPADES (Semantic PArsing of DEclarative Sentences) containing 93K cloze-style questions paired with answers. We evaluate all our models on this dataset. Our code and data are available at https://github.com/sivareddyg/graph-parser.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Text Readability and Simplification