Multiscale Generative Models: Improving Performance of a Generative Model Using Feedback from Other Dependent Generative Models

TL;DR

This paper introduces a hierarchical multi-GAN framework that models interactions in multi-agent systems and uses feedback from higher-level models to enhance lower-level generative models, improving realism and applicability.

Contribution

It proposes a novel multiscale GAN architecture with feedback mechanisms, enabling better modeling of complex interactions in multi-agent systems.

Findings

Feedback improves lower-level GAN performance

Hierarchical GANs capture multi-agent interactions effectively

Method applies across synthetic, time series, and image data

Abstract

Realistic fine-grained multi-agent simulation of real-world complex systems is crucial for many downstream tasks such as reinforcement learning. Recent work has used generative models (GANs in particular) for providing high-fidelity simulation of real-world systems. However, such generative models are often monolithic and miss out on modeling the interaction in multi-agent systems. In this work, we take a first step towards building multiple interacting generative models (GANs) that reflects the interaction in real world. We build and analyze a hierarchical set-up where a higher-level GAN is conditioned on the output of multiple lower-level GANs. We present a technique of using feedback from the higher-level GAN to improve performance of lower-level GANs. We mathematically characterize the conditions under which our technique is impactful, including understanding the transfer learning nature of our set-up. We present three distinct experiments on synthetic data, time series data, and image domain, revealing the wide applicability of our technique.