Drag-guided diffusion models for vehicle image generation

TL;DR

This paper introduces a physics-guided diffusion model that integrates engineering constraints into image generation, demonstrated by minimizing drag coefficients in vehicle images generated by Stable Diffusion.

Contribution

It proposes a novel physics-based guidance method for diffusion models, enabling optimization of engineering performance metrics during image synthesis.

Findings

Successfully integrated drag minimization into vehicle image generation

Enabled generation of vehicle images with optimized aerodynamic properties

Demonstrated potential for engineering design applications

Abstract

Denoising diffusion models trained at web-scale have revolutionized image generation. The application of these tools to engineering design is an intriguing possibility, but is currently limited by their inability to parse and enforce concrete engineering constraints. In this paper, we take a step towards this goal by proposing physics-based guidance, which enables optimization of a performance metric (as predicted by a surrogate model) during the generation process. As a proof-of-concept, we add drag guidance to Stable Diffusion, which allows this tool to generate images of novel vehicles while simultaneously minimizing their predicted drag coefficients.