Improved Instruction Ordering in Recipe-Grounded Conversation
Duong Minh Le, Ruohao Guo, Wei Xu, Alan Ritter

TL;DR
This paper addresses the challenge of correctly ordering instructions in recipe-grounded dialogue systems by introducing auxiliary tasks and a new dataset, revealing that even advanced models like ChatGPT still make ordering errors.
Contribution
The paper proposes auxiliary subtasks for user intent detection and instruction state tracking to improve instruction ordering in recipe dialogue systems, and introduces the ChattyChef dataset.
Findings
Incorporating intent and state tracking improves response accuracy.
ChatGPT makes 10.7% errors, with half being out-of-order instructions.
The ChattyChef dataset supports further research in instruction grounding.
Abstract
In this paper, we study the task of instructional dialogue and focus on the cooking domain. Analyzing the generated output of the GPT-J model, we reveal that the primary challenge for a recipe-grounded dialog system is how to provide the instructions in the correct order. We hypothesize that this is due to the model's lack of understanding of user intent and inability to track the instruction state (i.e., which step was last instructed). Therefore, we propose to explore two auxiliary subtasks, namely User Intent Detection and Instruction State Tracking, to support Response Generation with improved instruction grounding. Experimenting with our newly collected dataset, ChattyChef, shows that incorporating user intent and instruction state information helps the response generation model mitigate the incorrect order issue. Furthermore, to investigate whether ChatGPT has completely solved…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Speech and dialogue systems · Intelligent Tutoring Systems and Adaptive Learning
MethodsFocus
