Distributional Correlation--Aware Knowledge Distillation for Stock Trading Volume Prediction

TL;DR

This paper introduces a novel distributional correlation-aware knowledge distillation framework for stock trading volume prediction, enabling a lightweight model to learn from a probabilistic teacher and pairwise correlations, outperforming baselines.

Contribution

The paper proposes a new distillation method for regression tasks that incorporates distributional and correlation information, improving stock volume prediction accuracy and model compression.

Findings

Achieves 5x model size reduction while maintaining 99.6% accuracy.

Outperforms vanilla distillation methods in low-resource scenarios.

Effective in real-world stock trading volume datasets.

Abstract

Traditional knowledge distillation in classification problems transfers the knowledge via class correlations in the soft label produced by teacher models, which are not available in regression problems like stock trading volume prediction. To remedy this, we present a novel distillation framework for training a light-weight student model to perform trading volume prediction given historical transaction data. Specifically, we turn the regression model into a probabilistic forecasting model, by training models to predict a Gaussian distribution to which the trading volume belongs. The student model can thus learn from the teacher at a more informative distributional level, by matching its predicted distributions to that of the teacher. Two correlational distillation objectives are further introduced to encourage the student to produce consistent pair-wise relationships with the teacher model. We evaluate the framework on a real-world stock volume dataset with two different time window settings. Experiments demonstrate that our framework is superior to strong baseline models, compressing the model size by $5\times$ while maintaining $99.6\%$ prediction accuracy. The extensive analysis further reveals that our framework is more effective than vanilla distillation methods under low-resource scenarios.