LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content

TL;DR

LlamaLens is a novel multilingual LLM specifically designed for analyzing news and social media content, outperforming existing models on multiple datasets across Arabic, English, and Hindi, addressing a significant gap in domain-specific multilingual NLP.

Contribution

This paper introduces LlamaLens, the first specialized multilingual LLM for news and social media analysis, focusing on domain-specificity and multilingual capabilities, with extensive evaluation across 18 tasks and 52 datasets.

Findings

LlamaLens outperforms SOTA on 23 datasets

Achieves comparable performance on 8 datasets

Models and resources are publicly available

Abstract

Large Language Models (LLMs) have demonstrated remarkable success as general-purpose task solvers across various fields. However, their capabilities remain limited when addressing domain-specific problems, particularly in downstream NLP tasks. Research has shown that models fine-tuned on instruction-based downstream NLP datasets outperform those that are not fine-tuned. While most efforts in this area have primarily focused on resource-rich languages like English and broad domains, little attention has been given to multilingual settings and specific domains. To address this gap, this study focuses on developing a specialized LLM, LlamaLens, for analyzing news and social media content in a multilingual context. To the best of our knowledge, this is the first attempt to tackle both domain specificity and multilinguality, with a particular focus on news and social media. Our experimental setup includes 18 tasks, represented by 52 datasets covering Arabic, English, and Hindi. We demonstrate that LlamaLens outperforms the current state-of-the-art (SOTA) on 23 testing sets, and achieves comparable performance on 8 sets. We make the models and resources publicly available for the research community (https://huggingface.co/collections/QCRI/llamalens-672f7e0604a0498c6a2f0fe9).