Physical Context and Timing Aware Sequence Generating GANs

TL;DR

This paper introduces PCTGAN, a novel GAN model that generates sequential images considering physical contexts and specific timing, improving realism and temporal accuracy in generated sequences.

Contribution

The paper presents PCTGAN, a GAN that incorporates physical context and timing awareness for sequential image generation, addressing limitations of existing models.

Findings

PCTGAN effectively generates images with accurate physical contexts.

Timing information improves the realism of generated sequences.

Physical context consideration enhances the quality of sequential image generation.

Abstract

Generative Adversarial Networks (GANs) have shown remarkable successes in generating realistic images and interpolating changes between images. Existing models, however, do not take into account physical contexts behind images in generating the images, which may cause unrealistic changes. Furthermore, it is difficult to generate the changes at a specific timing and they often do not match with actual changes. This paper proposes a novel GAN, named Physical Context and Timing aware sequence generating GANs (PCTGAN), that generates an image in a sequence at a specific timing between two images with considering physical contexts behind them. Our method consists of three components: an encoder, a generator, and a discriminator. The encoder estimates latent vectors from the beginning and ending images, their timings, and a target timing. The generator generates images and the physical contexts at the beginning, ending, and target timing from the corresponding latent vectors. The discriminator discriminates whether the generated images and contexts are real or not. In the experiments, PCTGAN is applied to a data set of sequential changes of shapes in die forging processes. We show that both timing and physical contexts are effective in generating sequential images.