Domestic sound event detection by shift consistency mean-teacher training and adversarial domain adaptation

TL;DR

This paper enhances domestic sound event detection by refining semi-supervised training with shift consistency and adversarial domain adaptation, achieving a new state-of-the-art F1 score on a benchmark dataset.

Contribution

It demonstrates that removing interpolation consistency training and integrating shift consistency with adversarial domain adaptation improves detection accuracy.

Findings

Achieved 47.2% F1 score on DCASE 2020 task 4 dataset.

Removing ICT prevents distribution divergence between synthetic and real data.

Integrating SCT with ADA yields better domain adaptation and detection performance.

Abstract

Semi-supervised learning and domain adaptation techniques have drawn increasing attention in the field of domestic sound event detection thanks to the availability of large amounts of unlabeled data and the relative ease to generate synthetic strongly-labeled data. In a previous work, several semi-supervised learning strategies were designed to boost the performance of a mean-teacher model. Namely, these strategies include shift consistency training (SCT), interpolation consistency training (ICT), and pseudo-labeling. However, adversarial domain adaptation (ADA) did not seem to improve the event detection accuracy further when we attempt to compensate for the domain gap between synthetic and real data. In this research, we empirically found that ICT tends to pull apart the distributions of synthetic and real data in t-SNE plots. Therefore, ICT is abandoned while SCT, in contrast, is applied to train both the student and the teacher models. With these modifications, the system successfully integrates with an ADA network, and we achieve 47.2% in the F1 score on the DCASE 2020 task 4 dataset, which is 2.1% higher than what was reported in the previous work.