CAP-LLM: Context-Augmented Personalized Large Language Models for News Headline Generation

TL;DR

CAP-LLM is a novel framework that enhances personalized news headline generation by integrating user preferences and factual consistency constraints into large language models, achieving state-of-the-art results.

Contribution

It introduces a comprehensive framework combining user interest encoding, context injection, and fact reinforcement to improve personalization and factual accuracy in headline generation.

Findings

Significantly improves factual consistency (FactCC 87.50)

Enhances personalization metrics (Pc(avg) 2.73, Pc(max) 17.25)

Achieves state-of-the-art performance on PENS dataset

Abstract

In the era of information overload, personalized news headline generation is crucial for engaging users by tailoring content to their preferences while accurately conveying news facts. Existing methods struggle with effectively capturing complex user interests and ensuring factual consistency, often leading to generic or misleading headlines. Leveraging the unprecedented capabilities of Large Language Models (LLMs) in text generation, we propose Context-Augmented Personalized LLM (CAP-LLM), a novel framework that integrates user preferences and factual consistency constraints into a powerful pre-trained LLM backbone. CAP-LLM features a User Preference Encoder to capture long-term user interests, a Context Injection Adapter to seamlessly integrate these preferences and current article context into the LLM's generation process, and a Fact-Consistency Reinforcement Module employing a novel contrastive loss to mitigate hallucination. Evaluated on the real-world PENS dataset, CAP-LLM achieves state-of-the-art performance across all metrics. Notably, it significantly improves factual consistency (FactCC of 87.50) over strong baselines like BART (86.67), while simultaneously enhancing personalization (Pc(avg) 2.73, Pc(max) 17.25) and content coverage (ROUGE-1 26.55, ROUGE-2 9.95, ROUGE-L 23.01). Our ablation studies, human evaluations, and sensitivity analyses further validate the effectiveness of each component and the robustness of our approach, demonstrating CAP-LLM's ability to achieve a superior balance between personalization and factual accuracy in news headline generation.