Accelerating Deep Learning with Fixed Time Budget

TL;DR

This paper introduces a novel training technique that enables deep learning models to be trained effectively within fixed time budgets by leveraging sample importance and dynamic ranking, improving performance in vision tasks.

Contribution

It presents a new method for efficient deep learning training under time constraints, addressing practical limitations in edge and federated learning scenarios.

Findings

Consistent performance improvements across classification and regression tasks.

Effective in various state-of-the-art deep learning models.

Significant gains in training efficiency within fixed time budgets.

Abstract

The success of modern deep learning is attributed to two key elements: huge amounts of training data and large model sizes. Where a vast amount of data allows the model to learn more features, the large model architecture boosts the learning capability of the model. However, both these factors result in prolonged training time. In some practical applications such as edge-based learning and federated learning, limited-time budgets necessitate more efficient training methods. This paper proposes an effective technique for training arbitrary deep learning models within fixed time constraints utilizing sample importance and dynamic ranking. The proposed method is extensively evaluated in both classification and regression tasks in computer vision. The results consistently show clear gains achieved by the proposed method in improving the learning performance of various state-of-the-art deep learning models in both regression and classification tasks.