Contrastive Pre-training for Imbalanced Corporate Credit Ratings

TL;DR

This paper introduces CP4CCR, a contrastive pre-training framework that leverages self-supervised learning to address class imbalance in corporate credit rating, significantly improving model performance on imbalanced datasets.

Contribution

The paper presents a novel contrastive pre-training approach with self-supervised tasks to enhance corporate credit rating models under class imbalance conditions.

Findings

Improves credit rating accuracy for minority classes

Enhances model initialization for better downstream performance

Effective on Chinese corporate rating dataset

Abstract

Corporate credit rating reflects the level of corporate credit and plays a crucial role in modern financial risk control. But real-world credit rating data usually shows long-tail distributions, which means heavy class imbalanced problem challenging the corporate credit rating system greatly. To tackle that, inspried by the recent advances of pre-train techniques in self-supervised representation learning, we propose a novel framework named Contrastive Pre-training for Corporate Credit Rating (CP4CCR), which utilizes the self-surpervision for getting over class imbalance. Specifically, we propose to, in the first phase, exert constrastive self-superivised pre-training without label information, which want to learn a better class-agnostic initialization. During this phase, two self-supervised task are developed within CP4CCR: (i) Feature Masking (FM) and (ii) Feature Swapping(FS). In the second phase, we can train any standard corporate redit rating model initialized by the pre-trained network. Extensive experiments conducted on the Chinese public-listed corporate rating dataset, prove that CP4CCR can improve the performance of standard corporate credit rating models, especially for class with few samples.