Dual-Criterion Curriculum Learning: Application to Temporal Data

TL;DR

This paper introduces a dual-criterion curriculum learning framework that combines loss-based and density-based difficulty assessments to improve training efficiency on time-series forecasting tasks.

Contribution

The novel DCCL framework integrates two difficulty measures, addressing the challenge of defining meaningful difficulty criteria in curriculum learning.

Findings

Density-based curricula outperform loss-only methods.

Hybrid curricula improve time-series forecasting accuracy.

Dual-criterion approach enhances training efficiency.

Abstract

Curriculum Learning (CL) is a meta-learning paradigm that trains a model by feeding the data instances incrementally according to a schedule, which is based on difficulty progression. Defining meaningful difficulty assessment measures is crucial and most usually the main bottleneck for effective learning, while also in many cases the employed heuristics are only application-specific. In this work, we propose the Dual-Criterion Curriculum Learning (DCCL) framework that combines two views of assessing instance-wise difficulty: a loss-based criterion is complemented by a density-based criterion learned in the data representation space. Essentially, DCCL calibrates training-based evidence (loss) under the consideration that data sparseness amplifies the learning difficulty. As a testbed, we choose the time-series forecasting task. We evaluate our framework on multivariate time-series benchmarks under standard One-Pass and Baby-Steps training schedules. Empirical results show the interest of density-based and hybrid dual-criterion curricula over loss-only baselines and standard non-CL training in this setting.