Minimum Bayes' Risk Decoding for System Combination of Grammatical Error Correction Systems
Vyas Raina, Mark Gales

TL;DR
This paper introduces a novel Minimum Bayes' Risk decoding method tailored for grammatical error correction systems, improving system combination by aligning decoding with F-score evaluation and demonstrating enhanced performance across datasets.
Contribution
The paper proposes a new MBR loss function directly related to GEC evaluation criteria and an expanded candidate set generation method, advancing system combination techniques.
Findings
MBR decoding improves GEC system combination performance.
Varying reward metrics allows control over precision, recall, and F-score.
Experimental results show state-of-the-art performance on multiple datasets.
Abstract
For sequence-to-sequence tasks it is challenging to combine individual system outputs. Further, there is also often a mismatch between the decoding criterion and the one used for assessment. Minimum Bayes' Risk (MBR) decoding can be used to combine system outputs in a manner that encourages better alignment with the final assessment criterion. This paper examines MBR decoding for Grammatical Error Correction (GEC) systems, where performance is usually evaluated in terms of edits and an associated F-score. Hence, we propose a novel MBR loss function directly linked to this form of criterion. Furthermore, an approach to expand the possible set of candidate sentences is described. This builds on a current max-voting combination scheme, as well as individual edit-level selection. Experiments on three popular GEC datasets and with state-of-the-art GEC systems demonstrate the efficacy of the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Educational Technology and Assessment
