RepControlNet: ControlNet Reparameterization
Zhaoli Deng, Kaibin Zhou, Fanyi Wang, Zhenpeng Mi

TL;DR
RepControlNet introduces a reparameterization approach for diffusion models that enables controllable generation without additional computational costs, improving efficiency and performance over existing methods.
Contribution
This paper presents RepControlNet, a novel reparameterization technique that achieves controllable diffusion model generation without increasing parameters or computation.
Findings
Effective control without extra parameters
Surpasses ControlNet in efficiency and performance
Validated on SD1.5 and SDXL datasets
Abstract
With the wide application of diffusion model, the high cost of inference resources has became an important bottleneck for its universal application. Controllable generation, such as ControlNet, is one of the key research directions of diffusion model, and the research related to inference acceleration and model compression is more important. In order to solve this problem, this paper proposes a modal reparameterization method, RepControlNet, to realize the controllable generation of diffusion models without increasing computation. In the training process, RepControlNet uses the adapter to modulate the modal information into the feature space, copy the CNN and MLP learnable layers of the original diffusion model as the modal network, and initialize these weights based on the original weights and coefficients. The training process only optimizes the parameters of the modal network. In the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications
MethodsAdapter · Diffusion
