Tracking of Intermittent and Moving Speakers : Dataset and Metrics

TL;DR

This paper addresses the challenge of tracking intermittent and moving speakers, introducing a new dataset and metrics to evaluate track identity maintenance during discontinuous and dynamic speaker movements.

Contribution

It introduces LibriJump, a novel dataset for speaker tracking with changing positions during silence, and proposes adapted association metrics for better performance evaluation.

Findings

Association metrics complement existing tracking metrics.

Discontinuous tracks pose unique challenges for speaker tracking.

LibriJump enables evaluation of tracking methods on dynamic, intermittent speaker movements.

Abstract

This paper presents the problem of tracking intermittent and moving sources, i.e, sources that may change position when they are inactive. This issue is seldom explored, and most current tracking methods rely on spatial observations for track identity management. They are either based on a previous localization step, or designed to perform joint localization and tracking by predicting ordered position estimates. This raises concerns about whether such methods can maintain reliable track identity assignment performance when dealing with discontinuous spatial tracks, which may be caused by a change of direction during silence. We introduce LibriJump, a novel dataset of acoustic scenes in the First Order Ambisonics format focusing on speaker tracking. The dataset contains speakers with changing positions during inactivity periods, thus simulating discontinuous tracks. To measure the identity assignment performance, we propose to use tracking association metrics adapted from the computer vision community. We provide experiments showing the complementarity of association metrics with previously used tracking metrics, given continuous and discontinuous spatial tracks.