A time-scale modification dataset with subjective quality labels

TL;DR

This paper introduces a comprehensive dataset with subjective quality labels for Time Scale Modification (TSM), enabling the development and evaluation of objective quality measures, and provides initial results demonstrating the dataset's utility.

Contribution

The paper presents a new, large-scale dataset with subjective quality ratings for TSM, including diverse audio types and extensive listener evaluations, to improve objective quality assessment methods.

Findings

Objective measures poorly correlate with subjective quality.

Listeners' ratings are consistent across demographics.

Initial retrained objective measure shows promising results.

Abstract

Time Scale Modification (TSM) is a well-researched field; however, no effective objective measure of quality exists. This paper details the creation, subjective evaluation, and analysis of a dataset for use in the development of an objective measure of quality for TSM. Comprised of two parts, the training component contains 88 source files processed using six TSM methods at 10 time scales, while the testing component contains 20 source files processed using three additional methods at four time scales. The source material contains speech, solo harmonic and percussive instruments, sound effects, and a range of music genres. Ratings (42 529) were collected from 633 sessions using laboratory and remote collection methods. Analysis of results shows no correlation between age and quality of rating; expert and non-expert listeners to be equivalent; minor differences between participants with and without hearing issues; and minimal differences between testing modalities. A comparison of published objective measures and subjective scores shows the objective measures to be poor indicators of subjective quality. Initial results for a retrained objective measure of quality are presented with results approaching average root mean squared error loss and Pearson correlation values of subjective sessions. The labeled dataset is available at http://ieee-dataport.org/1987.