Recursively Summarizing Books with Human Feedback
Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe,, Jan Leike, Paul Christiano

TL;DR
This paper introduces a recursive, human-feedback-driven approach to summarizing entire books, enabling models to generate high-quality summaries of lengthy texts efficiently.
Contribution
It presents a novel recursive summarization method combining human feedback with model-assisted decomposition, achieving state-of-the-art results on book-length summarization and question-answering benchmarks.
Findings
Model produces summaries comparable to human-written ones (~5%)
Achieves state-of-the-art results on BookSum dataset
Zero-shot QA performance on NarrativeQA is improved
Abstract
A major challenge for scaling machine learning is training models to perform tasks that are very difficult or time-consuming for humans to evaluate. We present progress on this problem on the task of abstractive summarization of entire fiction novels. Our method combines learning from human feedback with recursive task decomposition: we use models trained on smaller parts of the task to assist humans in giving feedback on the broader task. We collect a large volume of demonstrations and comparisons from human labelers, and fine-tune GPT-3 using behavioral cloning and reward modeling to do summarization recursively. At inference time, the model first summarizes small sections of the book and then recursively summarizes these summaries to produce a summary of the entire book. Our human labelers are able to supervise and evaluate the models quickly, despite not having read the entire books…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
[ML News] Plagiarism Case w/ Plot Twist | CLIP for video surveillance | OpenAI summarizes books· youtube
Taxonomy
TopicsTopic Modeling · Advanced Text Analysis Techniques · Natural Language Processing Techniques
MethodsAttention Is All You Need · Linear Layer · Cosine Annealing · Attention Dropout · Adam · 15 Ways to Contact How can i speak to someone at Delta Airlines · Byte Pair Encoding · Dropout · {Dispute@FaQ-s}How to file a dispute with Expedia? · Linear Warmup With Cosine Annealing
