GateNLP at SemEval-2025 Task 10: Hierarchical Three-Step Prompting for Multilingual Narrative Classification

TL;DR

This paper introduces Hierarchical Three-Step Prompting (H3Prompt), a novel LLM-based method for multilingual narrative classification, achieving top results in SemEval-2025 Task 10 for English news articles.

Contribution

It presents a new hierarchical prompting strategy for multilingual narrative classification, improving accuracy over existing methods.

Findings

Achieved top performance on English test set among 28 teams.

Effective hierarchical prompting for multi-level narrative classification.

Code available for reproducibility and further research.

Abstract

The proliferation of online news and the increasing spread of misinformation necessitate robust methods for automatic data analysis. Narrative classification is emerging as a important task, since identifying what is being said online is critical for fact-checkers, policy markers and other professionals working on information studies. This paper presents our approach to SemEval 2025 Task 10 Subtask 2, which aims to classify news articles into a pre-defined two-level taxonomy of main narratives and sub-narratives across multiple languages. We propose Hierarchical Three-Step Prompting (H3Prompt) for multilingual narrative classification. Our methodology follows a three-step Large Language Model (LLM) prompting strategy, where the model first categorises an article into one of two domains (Ukraine-Russia War or Climate Change), then identifies the most relevant main narratives, and finally assigns sub-narratives. Our approach secured the top position on the English test set among 28 competing teams worldwide. The code is available at https://github.com/GateNLP/H3Prompt.