Label Encoding for Regression Networks

TL;DR

This paper introduces binary-encoded labels (BEL) for regression neural networks, improving accuracy by converting regression into a classification problem with robust encoding/decoding strategies, outperforming existing methods on multiple benchmarks.

Contribution

The paper proposes a novel binary encoding framework for regression tasks, including encoding/decoding functions and training losses, achieving state-of-the-art results across various benchmarks.

Findings

BEL outperforms direct regression methods.

Encoding strategies balance error probability and correction.

Framework is compatible with diverse architectures and metrics.

Abstract

Deep neural networks are used for a wide range of regression problems. However, there exists a significant gap in accuracy between specialized approaches and generic direct regression in which a network is trained by minimizing the squared or absolute error of output labels. Prior work has shown that solving a regression problem with a set of binary classifiers can improve accuracy by utilizing well-studied binary classification algorithms. We introduce binary-encoded labels (BEL), which generalizes the application of binary classification to regression by providing a framework for considering arbitrary multi-bit values when encoding target values. We identify desirable properties of suitable encoding and decoding functions used for the conversion between real-valued and binary-encoded labels based on theoretical and empirical study. These properties highlight a tradeoff between classification error probability and error-correction capabilities of label encodings. BEL can be combined with off-the-shelf task-specific feature extractors and trained end-to-end. We propose a series of sample encoding, decoding, and training loss functions for BEL and demonstrate they result in lower error than direct regression and specialized approaches while being suitable for a diverse set of regression problems, network architectures, and evaluation metrics. BEL achieves state-of-the-art accuracies for several regression benchmarks. Code is available at https://github.com/ubc-aamodt-group/BEL_regression.