Foodfusion: A Novel Approach for Food Image Composition via Diffusion Models

TL;DR

Foodfusion introduces a new diffusion-based method and a large dataset for realistic food image composition, effectively fusing foreground and background images to generate natural food images.

Contribution

The paper presents a novel diffusion model-based food image composition method and a high-quality dataset, addressing previous limitations in multi-image fusion and dataset availability.

Findings

Effective fusion of foreground and background images demonstrated

High-quality dataset FC22k enables better training and evaluation

Scalable approach suitable for diverse food image synthesis

Abstract

Food image composition requires the use of existing dish images and background images to synthesize a natural new image, while diffusion models have made significant advancements in image generation, enabling the construction of end-to-end architectures that yield promising results. However, existing diffusion models face challenges in processing and fusing information from multiple images and lack access to high-quality publicly available datasets, which prevents the application of diffusion models in food image composition. In this paper, we introduce a large-scale, high-quality food image composite dataset, FC22k, which comprises 22,000 foreground, background, and ground truth ternary image pairs. Additionally, we propose a novel food image composition method, Foodfusion, which leverages the capabilities of the pre-trained diffusion models and incorporates a Fusion Module for processing and integrating foreground and background information. This fused information aligns the foreground features with the background structure by merging the global structural information at the cross-attention layer of the denoising UNet. To further enhance the content and structure of the background, we also integrate a Content-Structure Control Module. Extensive experiments demonstrate the effectiveness and scalability of our proposed method.