# The 2022 NIST Language Recognition Evaluation

## TL;DR

The 2022 NIST Language Recognition Evaluation assessed the performance of various systems on conversational and broadcast speech, emphasizing African languages and variable speech durations to advance language recognition technology.

## Contribution

This paper provides an overview and analysis of the latest NIST language recognition evaluation, introducing new evaluation features and insights into system performance across languages and durations.

## Key findings

- Oromo and Tigrinya are easier to detect.
- Xhosa and Zulu are more challenging.
- Performance improves with longer speech segments up to a point.

## Abstract

In 2022, the U.S. National Institute of Standards and Technology (NIST) conducted the latest Language Recognition Evaluation (LRE) in an ongoing series administered by NIST since 1996 to foster research in language recognition and to measure state-of-the-art technology. Similar to previous LREs, LRE22 focused on conversational telephone speech (CTS) and broadcast narrowband speech (BNBS) data. LRE22 also introduced new evaluation features, such as an emphasis on African languages, including low resource languages, and a test set consisting of segments containing between 3s and 35s of speech randomly sampled and extracted from longer recordings. A total of 21 research organizations, forming 16 teams, participated in this 3-month long evaluation and made a total of 65 valid system submissions to be evaluated. This paper presents an overview of LRE22 and an analysis of system performance over different evaluation conditions. The evaluation results suggest that Oromo and Tigrinya are easier to detect while Xhosa and Zulu are more challenging. A greater confusability is seen for some language pairs. When speech duration increased, system performance significantly increased up to a certain duration, and then a diminishing return on system performance is observed afterward.