Cost-aware Pre-training for Multiclass Cost-sensitive Deep Learning

TL;DR

This paper introduces a novel cost-aware pre-training method for deep learning that incorporates misclassification costs during both pre-training and training, leading to improved performance in cost-sensitive tasks.

Contribution

It proposes a new algorithm that effectively integrates cost information into the pre-training stage of deep learning models, enhancing their ability to handle cost-sensitive classification.

Findings

Outperforms models without cost-aware pre-training

Effective incorporation of cost information during pre-training improves accuracy

Demonstrates significant gains across multiple experimental setups

Abstract

Deep learning has been one of the most prominent machine learning techniques nowadays, being the state-of-the-art on a broad range of applications where automatic feature extraction is needed. Many such applications also demand varying costs for different types of mis-classification errors, but it is not clear whether or how such cost information can be incorporated into deep learning to improve performance. In this work, we propose a novel cost-aware algorithm that takes into account the cost information into not only the training stage but also the pre-training stage of deep learning. The approach allows deep learning to conduct automatic feature extraction with the cost information effectively. Extensive experimental results demonstrate that the proposed approach outperforms other deep learning models that do not digest the cost information in the pre-training stage.