Toward Zero-Shot Instruction Following
Renze Lou, Wenpeng Yin

TL;DR
This paper introduces a more realistic zero-shot instruction following setting that leverages task definitions and proposes strategies to improve model understanding, achieving state-of-the-art results on Super-NaturalInstructions.
Contribution
It presents a novel zero-shot instruction following framework using critical sentence identification and ranking objectives to enhance task comprehension from definitions.
Findings
Achieved state-of-the-art performance on Super-NaturalInstructions
Effective identification of critical sentences improves task understanding
Ranking objectives enhance model output accuracy
Abstract
This work proposes a challenging yet more realistic setting for zero-shot cross-task generalization: zero-shot instruction following, presuming the existence of a paragraph-style task definition while no demonstrations exist. To better learn the task supervision from the definition, we propose two strategies: first, to automatically find out the critical sentences in the definition; second, a ranking objective to force the model to generate the gold outputs with higher probabilities when those critical parts are highlighted in the definition. The joint efforts of the two strategies yield state-of-the-art performance on the Super-NaturalInstructions. Our code is available on GitHub.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
