Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art

TL;DR

This paper surveys recent advancements in multi-objective deep learning, discussing various methods, applications, and challenges across supervised, unsupervised, reinforcement, and generative models, highlighting the complexity and diversity of the field.

Contribution

It introduces a comprehensive taxonomy of multi-objective deep learning methods and reviews recent progress, applications, and challenges across different learning paradigms.

Findings

Taxonomy based on training algorithms and decision maker needs

Coverage of supervised, unsupervised, reinforcement, and generative models

Discussion of recent advancements and successful applications

Abstract

Simultaneously considering multiple objectives in machine learning has been a popular approach for several decades, with various benefits for multi-task learning, the consideration of secondary goals such as sparsity, or multicriteria hyperparameter tuning. However - as multi-objective optimization is significantly more costly than single-objective optimization - the recent focus on deep learning architectures poses considerable additional challenges due to the very large number of parameters, strong nonlinearities and stochasticity. This survey covers recent advancements in the area of multi-objective deep learning. We introduce a taxonomy of existing methods - based on the type of training algorithm as well as the decision maker's needs - before listing recent advancements, and also successful applications. All three main learning paradigms supervised learning, unsupervised learning and reinforcement learning are covered, and we also address the recently very popular area of generative modeling.