Learning Loss Functions for Semi-supervised Learning via Discriminative Adversarial Networks

TL;DR

This paper introduces discriminative adversarial networks (DAN), a novel framework that leverages two discriminators to improve semi-supervised learning and automatically learn effective loss functions for various prediction tasks.

Contribution

The paper presents a new DAN framework that extends GANs with two discriminators, enabling semi-supervised learning and automatic loss function learning for classification and ranking.

Findings

DAN significantly improves predictor performance with small labeled datasets.

Automatically learned loss functions outperform standard loss functions.

DAN is effective across multiple datasets and tasks.

Abstract

We propose discriminative adversarial networks (DAN) for semi-supervised learning and loss function learning. Our DAN approach builds upon generative adversarial networks (GANs) and conditional GANs but includes the key differentiator of using two discriminators instead of a generator and a discriminator. DAN can be seen as a framework to learn loss functions for predictors that also implements semi-supervised learning in a straightforward manner. We propose instantiations of DAN for two different prediction tasks: classification and ranking. Our experimental results on three datasets of different tasks demonstrate that DAN is a promising framework for both semi-supervised learning and learning loss functions for predictors. For all tasks, the semi-supervised capability of DAN can significantly boost the predictor performance for small labeled sets with minor architecture changes across tasks. Moreover, the loss functions automatically learned by DANs are very competitive and usually outperform the standard pairwise and negative log-likelihood loss functions for both semi-supervised and supervised learning.