Timers and Such: A Practical Benchmark for Spoken Language Understanding with Numbers

TL;DR

This paper presents Timers and Such, a new open source dataset of spoken English commands involving numbers, designed to improve spoken language understanding for voice control applications, along with baseline model experiments.

Contribution

The paper introduces a novel dataset, Timers and Such, filling a gap in spoken language understanding resources for number-related commands, and provides baseline model evaluations.

Findings

Baseline models achieve moderate accuracy on the dataset.

The dataset enables targeted evaluation of number understanding in speech models.

Open source code facilitates further research and development.

Abstract

This paper introduces Timers and Such, a new open source dataset of spoken English commands for common voice control use cases involving numbers. We describe the gap in existing spoken language understanding datasets that Timers and Such fills, the design and creation of the dataset, and experiments with a number of ASR-based and end-to-end baseline models, the code for which has been made available as part of the SpeechBrain toolkit.