Enhancing Accuracy in Generative Models via Knowledge Transfer

TL;DR

This paper introduces a novel transfer learning framework that leverages shared structures between tasks to improve the accuracy of generative models like diffusion and normalizing flows, demonstrating significant performance gains.

Contribution

It proposes a new transfer learning approach based on distribution metrics and shared embeddings, enhancing generative model accuracy across different tasks.

Findings

Enhanced performance of diffusion models through transfer learning

Improved normalizing flows in transfer settings

Theoretical validation of shared structure benefits

Abstract

This paper investigates the accuracy of generative models and the impact of knowledge transfer on their generation precision. Specifically, we examine a generative model for a target task, fine-tuned using a pre-trained model from a source task. Building on the "Shared Embedding" concept, which bridges the source and target tasks, we introduce a novel framework for transfer learning under distribution metrics such as the Kullback-Leibler divergence. This framework underscores the importance of leveraging inherent similarities between diverse tasks despite their distinct data distributions. Our theory suggests that the shared structures can augment the generation accuracy for a target task, reliant on the capability of a source model to identify shared structures and effective knowledge transfer from source to target learning. To demonstrate the practical utility of this framework, we explore the theoretical implications for two specific generative models: diffusion and normalizing flows. The results show enhanced performance in both models over their non-transfer counterparts, indicating advancements for diffusion models and providing fresh insights into normalizing flows in transfer and non-transfer settings. These results highlight the significant contribution of knowledge transfer in boosting the generation capabilities of these models.