Unified Backpropagation for Multi-Objective Deep Learning

TL;DR

This paper introduces a unified backpropagation framework that optimizes multiple hybrid objectives simultaneously in deep neural networks, improving classification performance across various datasets.

Contribution

It proposes a novel method to unify multi-objective optimization in deep learning using evidence theory, simplifying complex gradient formulations.

Findings

Consistent performance improvements across multiple datasets.

Effective integration of hybrid loss functions.

Simplified multi-objective optimization process.

Abstract

A common practice in most of deep convolutional neural architectures is to employ fully-connected layers followed by Softmax activation to minimize cross-entropy loss for the sake of classification. Recent studies show that substitution or addition of the Softmax objective to the cost functions of support vector machines or linear discriminant analysis is highly beneficial to improve the classification performance in hybrid neural networks. We propose a novel paradigm to link the optimization of several hybrid objectives through unified backpropagation. This highly alleviates the burden of extensive boosting for independent objective functions or complex formulation of multiobjective gradients. Hybrid loss functions are linked by basic probability assignment from evidence theory. We conduct our experiments for a variety of scenarios and standard datasets to evaluate the advantage of our proposed unification approach to deliver consistent improvements into the classification performance of deep convolutional neural networks.