Acting Flatterers via LLMs Sycophancy: Combating Clickbait with LLMs Opposing-Stance Reasoning

TL;DR

This paper introduces a novel framework that leverages LLMs' sycophantic tendencies to generate opposing reasoning pairs, improving clickbait detection by contrastive learning without relying on ground-truth labels.

Contribution

It proposes the SORG framework for generating high-quality opposing reasoning pairs and a new ORCD model that enhances clickbait detection through contrastive learning guided by LLM credibility scores.

Findings

Outperforms existing clickbait detection methods on benchmark datasets.

Effectively utilizes LLM-generated opposing reasoning for robust detection.

Demonstrates the advantage of contrastive learning with LLM-guided soft labels.

Abstract

The widespread proliferation of online content has intensified concerns about clickbait, deceptive or exaggerated headlines designed to attract attention. While Large Language Models (LLMs) offer a promising avenue for addressing this issue, their effectiveness is often hindered by Sycophancy, a tendency to produce reasoning that matches users' beliefs over truthful ones, which deviates from instruction-following principles. Rather than treating sycophancy as a flaw to be eliminated, this work proposes a novel approach that initially harnesses this behavior to generate contrastive reasoning from opposing perspectives. Specifically, we design a Self-renewal Opposing-stance Reasoning Generation (SORG) framework that prompts LLMs to produce high-quality agree and disagree reasoning pairs for a given news title without requiring ground-truth labels. To utilize the generated reasoning, we develop a local Opposing Reasoning-based Clickbait Detection (ORCD) model that integrates three BERT encoders to represent the title and its associated reasoning. The model leverages contrastive learning, guided by soft labels derived from LLM-generated credibility scores, to enhance detection robustness. Experimental evaluations on three benchmark datasets demonstrate that our method consistently outperforms LLM prompting, fine-tuned smaller language models, and state-of-the-art clickbait detection baselines.