Generative Optimization Networks for Memory Efficient Data Generation

TL;DR

This paper introduces GON, a memory-efficient generative framework that uses a discriminator and input space optimization, enabling effective data generation and anomaly detection on resource-limited devices.

Contribution

The paper proposes GON, a novel generative model without a generator, significantly reducing memory usage while maintaining high performance for data generation and anomaly detection.

Findings

GON achieves up to 32% higher detection F1 scores.

GON reduces memory consumption by 58%.

Training overhead increases by only 5%.

Abstract

In standard generative deep learning models, such as autoencoders or GANs, the size of the parameter set is proportional to the complexity of the generated data distribution. A significant challenge is to deploy resource-hungry deep learning models in devices with limited memory to prevent system upgrade costs. To combat this, we propose a novel framework called generative optimization networks (GON) that is similar to GANs, but does not use a generator, significantly reducing its memory footprint. GONs use a single discriminator network and run optimization in the input space to generate new data samples, achieving an effective compromise between training time and memory consumption. GONs are most suited for data generation problems in limited memory settings. Here we illustrate their use for the problem of anomaly detection in memory-constrained edge devices arising from attacks or intrusion events. Specifically, we use a GON to calculate a reconstruction-based anomaly score for input time-series windows. Experiments on a Raspberry-Pi testbed with two existing and a new suite of datasets show that our framework gives up to 32% higher detection F1 scores and 58% lower memory consumption, with only 5% higher training overheads compared to the state-of-the-art.