Gamified Speaker Comparison by Listening

TL;DR

This study explores how gamified listening interfaces influence naive listeners' ability to compare speakers, showing that gamification improves accuracy and error balance in speaker recognition tasks.

Contribution

It introduces and evaluates three different interface designs for speaker comparison tasks, demonstrating the benefits of gamification in enhancing listener performance.

Findings

Gamified interfaces improve listener accuracy in speaker comparison.

Gamification helps balance false alarms and misses.

Enhanced engagement correlates with better performance.

Abstract

We address speaker comparison by listening in a game-like environment, hypothesized to make the task more motivating for naive listeners. We present the same 30 trials selected with the help of an x-vector speaker recognition system from VoxCeleb to a total of 150 crowdworkers recruited through Amazon's Mechanical Turk. They are divided into cohorts of 50, each using one of three alternative interface designs: (i) a traditional (nongamified) design; (ii) a gamified design with feedback on decisions, along with points, game level indications, and possibility for interface customization; (iii) another gamified design with an additional constraint of maximum of 5 'lives' consumed by wrong answers. We analyze the impact of these interface designs to listener error rates (both misses and false alarms), probability calibration, time of quitting, along with survey questionnaire. The results indicate improved performance from (i) to (ii) and (iii), particularly in terms of balancing the two types of detection errors.