HumanACGAN: conditional generative adversarial network with human-based auxiliary classifier and its evaluation in phoneme perception

TL;DR

This paper introduces HumanACGAN, a novel conditional GAN model that incorporates human perceptual evaluations as both discriminator and auxiliary classifier, enabling the generation of phonemes aligned with human perception.

Contribution

The paper extends HumanGAN to HumanACGAN, allowing conditional data generation based on human perceptual acceptability, which was not possible with previous models.

Findings

Successfully trained a conditional generator for phoneme perception.

Demonstrated the effectiveness of human-based discriminator and classifier.

Validated the model's ability to produce human-acceptable phonemes.

Abstract

We propose a conditional generative adversarial network (GAN) incorporating humans' perceptual evaluations. A deep neural network (DNN)-based generator of a GAN can represent a real-data distribution accurately but can never represent a human-acceptable distribution, which are ranges of data in which humans accept the naturalness regardless of whether the data are real or not. A HumanGAN was proposed to model the human-acceptable distribution. A DNN-based generator is trained using a human-based discriminator, i.e., humans' perceptual evaluations, instead of the GAN's DNN-based discriminator. However, the HumanGAN cannot represent conditional distributions. This paper proposes the HumanACGAN, a theoretical extension of the HumanGAN, to deal with conditional human-acceptable distributions. Our HumanACGAN trains a DNN-based conditional generator by regarding humans as not only a discriminator but also an auxiliary classifier. The generator is trained by deceiving the human-based discriminator that scores the unconditioned naturalness and the human-based classifier that scores the class-conditioned perceptual acceptability. The training can be executed using the backpropagation algorithm involving humans' perceptual evaluations. Our experimental results in phoneme perception demonstrate that our HumanACGAN can successfully train this conditional generator.