Scaling HuBERT for African Languages: From Base to Large and XL

TL;DR

This paper introduces large-scale African speech models, SSA-HuBERT-Large and XL, trained solely on African audio, demonstrating that increased model size improves performance in ASR and LID tasks for Sub-Saharan languages.

Contribution

It presents the first large African speech models trained exclusively on African data, showing how increased capacity benefits low-resource language processing.

Findings

Larger models outperform smaller ones in ASR and LID tasks.

Models trained solely on African speech data improve performance.

Open weights are publicly released for research use.

Abstract

Despite recent progress in multilingual speech processing, African languages remain under-represented in both research and deployed systems, particularly when it comes to strong, open-weight encoders that transfer well under low-resource supervision. Self-supervised learning has proven especially promising in such settings, yet most publicly released models targeting African speech remain at BASE scale, leaving unanswered whether larger encoders, trained exclusively on Africa-centric audio, offer tangible benefits and how model capacity interacts with data composition. This work addresses that gap by introducing SSA-HuBERT-Large (317M parameters) and SSA-HuBERT-XL (964M parameters), the first large models trained solely on African speech, alongside a BASE size counterpart. We release these models as open weights: see https://huggingface.co/collections/Orange/african-speech-foundation-models. By conducting a carefully controlled experimental study focused exclusively on Sub-Saharan languages, covering automatic speech recognition (ASR) and language identification (LID) tasks, we demonstrate that larger architectures significantly improve performance by effectively leveraging large audio datasets.