Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning

TL;DR

This paper introduces Epochal Difficult Captions, a curriculum learning method that dynamically adjusts caption difficulty during training, improving automated audio captioning performance across multiple models.

Contribution

It presents a novel, lightweight curriculum learning approach that enhances audio captioning models without increasing training time.

Findings

Consistent performance improvements across three systems

Effective curriculum-based difficulty adjustment

Compatible with any model architecture

Abstract

In this paper, we propose an algorithm, Epochal Difficult Captions, to supplement the training of any model for the Automated Audio Captioning task. Epochal Difficult Captions is an elegant evolution to the keyword estimation task that previous work have used to train the encoder of the AAC model. Epochal Difficult Captions modifies the target captions based on a curriculum and a difficulty level determined as a function of current epoch. Epochal Difficult Captions can be used with any model architecture and is a lightweight function that does not increase training time. We test our results on three systems and show that using Epochal Difficult Captions consistently improves performance