SETGAN: Scale and Energy Trade-off GANs for Image Applications on Mobile Platforms

TL;DR

SETGAN introduces a client-server architecture that balances image quality and energy consumption for mobile image generation, enabling efficient, high-quality photo-realistic outputs on edge devices.

Contribution

The paper proposes SETGAN, a novel approach that trades off image accuracy for energy efficiency using a multi-scale GAN framework on mobile platforms.

Findings

Achieved 56% energy savings with 3-12% SSIM accuracy loss.

Enabled 4x faster training time on server with parallel multi-scale training.

Demonstrated effective image generation on mobile devices with energy constraints.

Abstract

We consider the task of photo-realistic unconditional image generation (generate high quality, diverse samples that carry the same visual content as the image) on mobile platforms using Generative Adversarial Networks (GANs). In this paper, we propose a novel approach to trade-off image generation accuracy of a GAN for the energy consumed (compute) at run-time called Scale-Energy Tradeoff GAN (SETGAN). GANs usually take a long time to train and consume a huge memory hence making it difficult to run on edge devices. The key idea behind SETGAN for an image generation task is for a given input image, we train a GAN on a remote server and use the trained model on edge devices. We use SinGAN, a single image unconditional generative model, that contains a pyramid of fully convolutional GANs, each responsible for learning the patch distribution at a different scale of the image. During the training process, we determine the optimal number of scales for a given input image and the energy constraint from the target edge device. Results show that with SETGAN's unique client-server-based architecture, we were able to achieve a 56% gain in energy for a loss of 3% to 12% SSIM accuracy. Also, with the parallel multi-scale training, we obtain around 4x gain in training time on the server.