The NTT DCASE2020 Challenge Task 6 system: Automated Audio Captioning with Keywords and Sentence Length Estimation

TL;DR

This paper presents a system for automated audio captioning that improves caption quality by estimating keywords and sentence length through multi-task learning, achieving significantly higher SPIDEr scores than the baseline.

Contribution

The novel approach simultaneously addresses word and sentence length indeterminacy in audio captioning using multi-task learning for keyword and sentence length estimation.

Findings

Achieved a SPIDEr score of 20.7, outperforming the baseline score of 5.4.

Demonstrated effectiveness of multi-task learning in resolving caption indeterminacy.

Validated the model on the DCASE 2020 Challenge dataset.

Abstract

This technical report describes the system participating to the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 Challenge, Task 6: automated audio captioning. Our submission focuses on solving two indeterminacy problems in automated audio captioning: word selection indeterminacy and sentence length indeterminacy. We simultaneously solve the main caption generation and sub indeterminacy problems by estimating keywords and sentence length through multi-task learning. We tested a simplified model of our submission using the development-testing dataset. Our model achieved 20.7 SPIDEr score where that of the baseline system was 5.4.