'John ate 5 apples' != 'John ate some apples': Self-Supervised Paraphrase Quality Detection for Algebraic Word Problems

TL;DR

This paper presents ParaQD, a self-supervised method for scoring the quality of paraphrases of algebraic word problems, improving paraphrase assessment and generation for educational purposes.

Contribution

The paper introduces a novel self-supervised approach with data augmentations for paraphrase quality detection in algebraic word problems, addressing limitations of existing models.

Findings

Outperforms state-of-the-art self-supervised methods by up to 32%

Demonstrates strong zero-shot performance

Effective in distinguishing high-quality from poor paraphrases

Abstract

This paper introduces the novel task of scoring paraphrases for Algebraic Word Problems (AWP) and presents a self-supervised method for doing so. In the current online pedagogical setting, paraphrasing these problems is helpful for academicians to generate multiple syntactically diverse questions for assessments. It also helps induce variation to ensure that the student has understood the problem instead of just memorizing it or using unfair means to solve it. The current state-of-the-art paraphrase generation models often cannot effectively paraphrase word problems, losing a critical piece of information (such as numbers or units) which renders the question unsolvable. There is a need for paraphrase scoring methods in the context of AWP to enable the training of good paraphrasers. Thus, we propose ParaQD, a self-supervised paraphrase quality detection method using novel data augmentations that can learn latent representations to separate a high-quality paraphrase of an algebraic question from a poor one by a wide margin. Through extensive experimentation, we demonstrate that our method outperforms existing state-of-the-art self-supervised methods by up to 32% while also demonstrating impressive zero-shot performance.