PROPEL: Probabilistic Parametric Regression Loss for Convolutional Neural Networks

TL;DR

PROPEL introduces a differentiable probabilistic regression loss for CNNs, enabling better uncertainty modeling and improved accuracy in multi-variate regression tasks like orientation estimation, with fewer parameters and enhanced generalization.

Contribution

It proposes a novel probabilistic regression loss, PROPEL, that allows CNNs to learn distribution parameters for regression, improving accuracy and efficiency over existing methods.

Findings

PROPEL improves CNN regression accuracy.

Reduces model parameters by 10x.

Enhances generalization in multi-variate regression.

Abstract

In recent years, Convolutional Neural Networks (CNNs) have enabled significant advancements to the state-of-the-art in computer vision. For classification tasks, CNNs have widely employed probabilistic output and have shown the significance of providing additional confidence for predictions. However, such probabilistic methodologies are not widely applicable for addressing regression problems using CNNs, as regression involves learning unconstrained continuous and, in many cases, multi-variate target variables. We propose a PRObabilistic Parametric rEgression Loss (PROPEL) that facilitates CNNs to learn parameters of probability distributions for addressing probabilistic regression problems. PROPEL is fully differentiable and, hence, can be easily incorporated for end-to-end training of existing CNN regression architectures using existing optimization algorithms. The proposed method is flexible as it enables learning complex unconstrained probabilities while being generalizable to higher dimensional multi-variate regression problems. We utilize a PROPEL-based CNN to address the problem of learning hand and head orientation from uncalibrated color images. Our experimental validation and comparison with existing CNN regression loss functions show that PROPEL improves the accuracy of a CNN by enabling probabilistic regression, while significantly reducing required model parameters by $10 \times$, resulting in improved generalization as compared to the existing state-of-the-art.