Imbalance in Balance: Online Concept Balancing in Generation Models

TL;DR

This paper introduces an online concept balancing method with a new loss function to improve stability and accuracy in visual generation models, addressing complex concept response issues.

Contribution

It proposes the IMBA loss function and a new benchmark for complex concepts, enhancing model responses with minimal code modifications.

Findings

Significant improvement in concept response capability

Effective on multiple test sets

Competitive results with minimal code changes

Abstract

In visual generation tasks, the responses and combinations of complex concepts often lack stability and are error-prone, which remains an under-explored area. In this paper, we attempt to explore the causal factors for poor concept responses through elaborately designed experiments. We also design a concept-wise equalization loss function (IMBA loss) to address this issue. Our proposed method is online, eliminating the need for offline dataset processing, and requires minimal code changes. In our newly proposed complex concept benchmark Inert-CompBench and two other public test sets, our method significantly enhances the concept response capability of baseline models and yields highly competitive results with only a few codes released at https://github.com/KwaiVGI/IMBA-Loss.