Quiz-Style Question Generation for News Stories
Adam D. Lelkes, Vinh Q. Tran, Cong Yu

TL;DR
This paper introduces a new dataset and methods for generating quiz-style questions from news articles to assess and improve news informedness, demonstrating effective model performance and positive user feedback.
Contribution
It presents the NewsQuizQA dataset and novel Transformer-based techniques for question-answer and distractor generation from news summaries.
Findings
Models outperform baselines on automated metrics.
Generated questions are rated as educational and enjoyable by users.
Weekly quizzes engaged real users over two months.
Abstract
A large majority of American adults get at least some of their news from the Internet. Even though many online news products have the goal of informing their users about the news, they lack scalable and reliable tools for measuring how well they are achieving this goal, and therefore have to resort to noisy proxy metrics (e.g., click-through rates or reading time) to track their performance. As a first step towards measuring news informedness at a scale, we study the problem of quiz-style multiple-choice question generation, which may be used to survey users about their knowledge of recent news. In particular, we formulate the problem as two sequence-to-sequence tasks: question-answer generation (QAG) and distractor, or incorrect answer, generation (DG). We introduce NewsQuizQA, the first dataset intended for quiz-style question-answer generation, containing 20K human written…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Expert finding and Q&A systems · Natural Language Processing Techniques
Methods7 Fastest Ways to Call American Airlines Reservations Number (USA Guide) · PEGASUS · Linear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Label Smoothing · Byte Pair Encoding · Dense Connections · Adam · Gated Linear Unit
