Robust Regression via Deep Negative Correlation Learning

TL;DR

This paper introduces a novel deep negative correlation learning approach for robust nonlinear regression, effectively combining ensemble diversity and accuracy without extra parameters, and demonstrating superior performance across various vision tasks.

Contribution

It generalizes negative correlation learning to deep regression, enabling efficient ensemble training with controlled bias-variance trade-off and reduced complexity.

Findings

Outperforms baseline methods on multiple vision tasks

Enables diverse and accurate deep regression ensembles

Reduces Rademacher Complexity for easier optimization

Abstract

Nonlinear regression has been extensively employed in many computer vision problems (e.g., crowd counting, age estimation, affective computing). Under the umbrella of deep learning, two common solutions exist i) transforming nonlinear regression to a robust loss function which is jointly optimizable with the deep convolutional network, and ii) utilizing ensemble of deep networks. Although some improved performance is achieved, the former may be lacking due to the intrinsic limitation of choosing a single hypothesis and the latter usually suffers from much larger computational complexity. To cope with those issues, we propose to regress via an efficient "divide and conquer" manner. The core of our approach is the generalization of negative correlation learning that has been shown, both theoretically and empirically, to work well for non-deep regression problems. Without extra parameters, the proposed method controls the bias-variance-covariance trade-off systematically and usually yields a deep regression ensemble where each base model is both "accurate" and "diversified". Moreover, we show that each sub-problem in the proposed method has less Rademacher Complexity and thus is easier to optimize. Extensive experiments on several diverse and challenging tasks including crowd counting, personality analysis, age estimation, and image super-resolution demonstrate the superiority over challenging baselines as well as the versatility of the proposed method.