GitSearch: Enhancing Community Notes Generation with Gap-Informed Targeted Search

TL;DR

GitSearch is a novel framework that improves community notes generation by identifying information gaps, retrieving relevant data, and synthesizing helpful notes, significantly outperforming existing methods and human contributions.

Contribution

This paper introduces GitSearch, a three-stage pipeline leveraging gap-informed targeted search for enhanced community notes generation, along with the PolBench benchmark for evaluation.

Findings

GitSearch achieves 99% coverage, nearly doubling the state-of-the-art.

GitSearch outperforms human-authored notes with a 69% win rate.

GitSearch produces higher helpfulness scores (3.87 vs. 3.36).

Abstract

Community-based moderation offers a scalable alternative to centralized fact-checking, yet it faces significant structural challenges, and existing AI-based methods fail in "cold start" scenarios. To tackle these challenges, we introduce GitSearch (Gap-Informed Targeted Search), a framework that treats human-perceived quality gaps, such as missing context, etc., as first-class signals. GitSearch has a three-stage pipeline: identifying information deficits, executing real-time targeted web-retrieval to resolve them, and synthesizing platform-compliant notes. To facilitate evaluation, we present PolBench, a benchmark of 78,698 U.S. political tweets with their associated Community Notes. We find GitSearch achieves 99% coverage, almost doubling coverage over the state-of-the-art. GitSearch surpasses human-authored helpful notes with a 69% win rate and superior helpfulness scores (3.87 vs. 3.36), demonstrating retrieval effectiveness that balanced the trade-off between scale and quality.