"This Suits You the Best": Query Focused Comparative Explainable Summarization

TL;DR

This paper introduces a novel task of generating query-focused comparative summaries for product recommendations using large language models, addressing the lack of dedicated datasets and improving inference efficiency.

Contribution

It proposes the QF-CES task, creates the MS-Q2P dataset, and leverages LLMs for personalized, category-agnostic summaries, reducing inference latency by 40%.

Findings

High correlation (0.74) with human judgments in evaluation.

MS-Q2P dataset enables effective training and evaluation.

LLMs produce coherent, query-specific comparative summaries.

Abstract

Product recommendations inherently involve comparisons, yet traditional opinion summarization often fails to provide holistic comparative insights. We propose the novel task of generating Query-Focused Comparative Explainable Summaries (QF-CES) using Multi-Source Opinion Summarization (M-OS). To address the lack of query-focused recommendation datasets, we introduce MS-Q2P, comprising 7,500 queries mapped to 22,500 recommended products with metadata. We leverage Large Language Models (LLMs) to generate tabular comparative summaries with query-specific explanations. Our approach is personalized, privacy-preserving, recommendation engine-agnostic, and category-agnostic. M-OS as an intermediate step reduces inference latency approximately by 40% compared to the direct input approach (DIA), which processes raw data directly. We evaluate open-source and proprietary LLMs for generating and assessing QF-CES. Extensive evaluations using QF-CES-PROMPT across 5 dimensions (clarity, faithfulness, informativeness, format adherence, and query relevance) showed an average Spearman correlation of 0.74 with human judgments, indicating its potential for QF-CES evaluation.