Teaching Human Behavior Improves Content Understanding Abilities Of LLMs
Somesh Singh, Harini S I, Yaman K Singla, Veeky Baths, Rajiv Ratn, Shah, Changyou Chen, Balaji Krishnamurthy

TL;DR
Training large language models on receiver behavior signals like likes and comments enhances their ability to understand content across various tasks, leveraging naturally collected data without additional annotation costs.
Contribution
This paper introduces a novel approach of using receiver behavior data to improve LLM content understanding, demonstrating significant performance gains across multiple benchmarks.
Findings
Improved performance on 46 video and image understanding tasks.
Outperforms many supervised baselines.
Uses freely available receiver behavior data for training.
Abstract
Communication is defined as "Who says what to whom with what effect". A message from a communicator generates downstream receiver effects, also known as behavior. Receiver behavior, being a downstream effect of the message, carries rich signals about it. Even after carrying signals about the message, the behavior data is often ignored while training large language models. We show that training LLMs on receiver behavior can actually help improve their content-understanding abilities. Specifically, we show that training LLMs to predict the receiver behavior of likes and comments improves the LLM's performance on a wide variety of downstream content understanding tasks. We show this performance increase over 46 video and image understanding tasks over 26 benchmark datasets across both 0-shot and fine-tuning settings, outperforming many supervised baselines. Moreover, since receiver…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsOpen Education and E-Learning
