Improving the Performance of Neural Networks in Regression Tasks Using Drawering

TL;DR

This paper introduces Drawering, a method to enhance neural network regression performance by modifying the learning process, which improves expressiveness and regularization without affecting evaluation speed.

Contribution

The proposed Drawering method improves regression neural networks by modifying the learning procedure, increasing expressiveness and regularization while maintaining evaluation efficiency.

Findings

Enhanced regression performance with Drawering

Better gradient behavior and regularization effects

No change in evaluation speed

Abstract

The method presented extends a given regression neural network to make its performance improve. The modification affects the learning procedure only, hence the extension may be easily omitted during evaluation without any change in prediction. It means that the modified model may be evaluated as quickly as the original one but tends to perform better. This improvement is possible because the modification gives better expressive power, provides better behaved gradients and works as a regularization. The knowledge gained by the temporarily extended neural network is contained in the parameters shared with the original neural network. The only cost is an increase in learning time.