QANA: LLM-based Question Generation and Network Analysis for Zero-shot Key Point Analysis and Beyond

TL;DR

QANA leverages large language models to generate questions from social media comments, constructs opinion networks, and identifies key points with flexible perspectives, achieving state-of-the-art zero-shot performance with reduced computational costs.

Contribution

It introduces a novel, flexible framework using LLMs for zero-shot opinion mining and key point analysis, improving efficiency and impartiality over previous methods.

Findings

QANA achieves comparable zero-shot performance to supervised models.

Questions with high centrality correlate with key points.

The framework reduces computational complexity from quadratic to linear.

Abstract

The proliferation of social media has led to information overload and increased interest in opinion mining. We propose "Question-Answering Network Analysis" (QANA), a novel opinion mining framework that utilizes Large Language Models (LLMs) to generate questions from users' comments, constructs a bipartite graph based on the comments' answerability to the questions, and applies centrality measures to examine the importance of opinions. We investigate the impact of question generation styles, LLM selections, and the choice of embedding model on the quality of the constructed QA networks by comparing them with annotated Key Point Analysis datasets. QANA achieves comparable performance to previous state-of-the-art supervised models in a zero-shot manner for Key Point Matching task, also reducing the computational cost from quadratic to linear. For Key Point Generation, questions with high PageRank or degree centrality align well with manually annotated key points. Notably, QANA enables analysts to assess the importance of key points from various aspects according to their selection of centrality measure. QANA's primary contribution lies in its flexibility to extract key points from a wide range of perspectives, which enhances the quality and impartiality of opinion mining.