HausaNLP: Current Status, Challenges and Future Directions for Hausa Natural Language Processing

TL;DR

This paper reviews the current state, challenges, and future directions of Hausa NLP, highlighting resource limitations and proposing strategic research avenues to advance language processing for Hausa, a low-resource language with millions of speakers.

Contribution

It provides a comprehensive overview of Hausa NLP, introduces a curated resource catalog, and discusses strategies to overcome challenges in model development and community engagement.

Findings

Hausa NLP has limited open-source datasets and model representations.

A curated catalog (HausaNLP) aggregates resources to facilitate research.

Identifies key challenges in integrating Hausa into large language models.

Abstract

Hausa Natural Language Processing (NLP) has gained increasing attention in recent years, yet remains understudied as a low-resource language despite having over 120 million first-language (L1) and 80 million second-language (L2) speakers worldwide. While significant advances have been made in high-resource languages, Hausa NLP faces persistent challenges, including limited open-source datasets and inadequate model representation. This paper presents an overview of the current state of Hausa NLP, systematically examining existing resources, research contributions, and gaps across fundamental NLP tasks: text classification, machine translation, named entity recognition, speech recognition, and question answering. We introduce HausaNLP (https://catalog.hausanlp.org), a curated catalog that aggregates datasets, tools, and research works to enhance accessibility and drive further development. Furthermore, we discuss challenges in integrating Hausa into large language models (LLMs), addressing issues of suboptimal tokenization and dialectal variation. Finally, we propose strategic research directions emphasizing dataset expansion, improved language modeling approaches, and strengthened community collaboration to advance Hausa NLP. Our work provides both a foundation for accelerating Hausa NLP progress and valuable insights for broader multilingual NLP research.