Skit-S2I: An Indian Accented Speech to Intent dataset

TL;DR

This paper introduces Skit-S2I, the first Indian-accented speech-to-intent dataset in the banking domain, enabling end-to-end SLU research for Indian accents and demonstrating the effectiveness of SSL pretrained representations.

Contribution

The paper releases the first Indian-accented speech-to-intent dataset in the banking domain and evaluates various pretrained speech encoders for intent classification.

Findings

SSL pretrained representations outperform ASR pretrained ones.

End-to-end SLU reduces latency and avoids cascading errors.

Dataset facilitates research on Indian-accented speech understanding.

Abstract

Conventional conversation assistants extract text transcripts from the speech signal using automatic speech recognition (ASR) and then predict intent from the transcriptions. Using end-to-end spoken language understanding (SLU), the intents of the speaker are predicted directly from the speech signal without requiring intermediate text transcripts. As a result, the model can optimize directly for intent classification and avoid cascading errors from ASR. The end-to-end SLU system also helps in reducing the latency of the intent prediction model. Although many datasets are available publicly for text-to-intent tasks, the availability of labeled speech-to-intent datasets is limited, and there are no datasets available in the Indian accent. In this paper, we release the Skit-S2I dataset, the first publicly available Indian-accented SLU dataset in the banking domain in a conversational tonality. We experiment with multiple baselines, compare different pretrained speech encoder's representations, and find that SSL pretrained representations perform slightly better than ASR pretrained representations lacking prosodic features for speech-to-intent classification. The dataset and baseline code is available at \url{https://github.com/skit-ai/speech-to-intent-dataset}