Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialogue
Lena Reed, Cecilia Li, Angela Ramirez, Liren Wu, and Marilyn Walker, (Natural Language, Dialogue Systems Lab, University of California, Santa, Cruz)

TL;DR
This paper demonstrates that few-shot prompt-based learning with Jurassic-1 significantly improves open-domain dialogue response quality and semantic accuracy, enabling generalization across domains and better control over dialogue acts.
Contribution
It introduces the first application of few-shot semantic prompt-based learning with Jurassic-1 for open-domain dialogue, showing improved response quality and domain generalization.
Findings
10-shot prompting yields significantly better coherence and semantic accuracy.
Jurassic-1 outperforms GPT-Neo in cross-domain semantic control.
Few-shot learning enables high-quality, semantically-controlled responses from meaning representations.
Abstract
One challenge with open-domain dialogue systems is the need to produce truthful, high-quality responses on any topic. We aim to improve the quality and coverage of Athena, an Alexa Prize dialogue system. We experiment with few-shot prompt-based learning, comparing GPT-Neo to Jurassic-1, for the movies, music, TV, sports, and video game domains, both within and cross-domain, with different prompt set sizes (2, 3, 10), formats, and meaning representations consisting of either sets of WikiData KG triples, or dialogue acts. Our evaluation uses BLEURT and human metrics, and shows that with 10-shot prompting, Athena-Jurassic's performance is significantly better for coherence and semantic accuracy. Experiments with 2-shot cross-domain prompts results in a huge performance drop for Athena-GPT-Neo, whose semantic accuracy falls to 0.41, and whose untrue hallucination rate increases to 12%.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Speech and dialogue systems
MethodsGPT-Neo
