Generative Model for Models: Rapid DNN Customization for Diverse Tasks and Resource Constraints

TL;DR

This paper presents NN-Factory, a generative framework that rapidly produces customized, lightweight deep learning models for diverse edge scenarios, significantly reducing customization time while maintaining high quality.

Contribution

The introduction of NN-Factory, a generative approach that directly creates task- and resource-specific models without retraining, enabling fast edge model customization.

Findings

Generates high-quality models within seconds.

Outperforms traditional customization methods in speed.

Effective across image classification and object detection tasks.

Abstract

Unlike cloud-based deep learning models that are often large and uniform, edge-deployed models usually demand customization for domain-specific tasks and resource-limited environments. Such customization processes can be costly and time-consuming due to the diversity of edge scenarios and the training load for each scenario. Although various approaches have been proposed for rapid resource-oriented customization and task-oriented customization respectively, achieving both of them at the same time is challenging. Drawing inspiration from the generative AI and the modular composability of neural networks, we introduce NN-Factory, an one-for-all framework to generate customized lightweight models for diverse edge scenarios. The key idea is to use a generative model to directly produce the customized models, instead of training them. The main components of NN-Factory include a modular supernet with pretrained modules that can be conditionally activated to accomplish different tasks and a generative module assembler that manipulate the modules according to task and sparsity requirements. Given an edge scenario, NN-Factory can efficiently customize a compact model specialized in the edge task while satisfying the edge resource constraints by searching for the optimal strategy to assemble the modules. Based on experiments on image classification and object detection tasks with different edge devices, NN-Factory is able to generate high-quality task- and resource-specific models within few seconds, faster than conventional model customization approaches by orders of magnitude.