Reasoning Circuits: Few-shot Multihop Question Generation with Structured Rationales
Saurabh Kulshreshtha, Anna Rumshisky

TL;DR
This paper introduces a structured rationale framework for multi-hop question generation that improves performance and control in low-shot settings using modest model sizes, reducing the need for extensive rationale annotation.
Contribution
The work presents a new multi-step rationale schema and a low-supervision approach for multi-hop question generation, enabling better control and performance with smaller models.
Findings
Improved question quality with structured rationales in low-shot regimes
Enhanced control over question difficulty through rationale schema
Better automatic and human evaluation results compared to baselines
Abstract
Multi-hop Question Generation is the task of generating questions which require the reader to reason over and combine information spread across multiple passages using several reasoning steps. Chain-of-thought rationale generation has been shown to improve performance on multi-step reasoning tasks and make model predictions more interpretable. However, few-shot performance gains from including rationales have been largely observed only in +100B language models, and otherwise require large scale manual rationale annotation. In this work, we introduce a new framework for applying chain-of-thought inspired structured rationale generation to multi-hop question generation under a very low supervision regime (8- to 128-shot). We propose to annotate a small number of examples following our proposed multi-step rationale schema, treating each reasoning step as a separate task to be performed by…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
