InteractPro: A Unified Framework for Motion-Aware Image Composition

TL;DR

InteractPro is a unified framework that combines simulation and diffusion methods, guided by an intelligent planner, to produce realistic, motion-aware image compositions with minimal manual intervention.

Contribution

It introduces InteractPlan, a large vision language model-based planner, and integrates simulation and diffusion modules for dynamic image composition, overcoming static output limitations.

Findings

Produces controllable, realistic motion effects in compositions

Outperforms traditional static composition methods

Effective across diverse scenarios

Abstract

We introduce InteractPro, a comprehensive framework for dynamic motion-aware image composition. At its core is InteractPlan, an intelligent planner that leverages a Large Vision Language Model (LVLM) for scenario analysis and object placement, determining the optimal composition strategy to achieve realistic motion effects. Based on each scenario, InteractPlan selects between our two specialized modules: InteractPhys and InteractMotion. InteractPhys employs an enhanced Material Point Method (MPM)-based simulation to produce physically faithful and controllable object-scene interactions, capturing diverse and abstract events that require true physical modeling. InteractMotion, in contrast, is a training-free method based on pretrained video diffusion. Traditional composition approaches suffer from two major limitations: requiring manual planning for object placement and generating static, motionless outputs. By unifying simulation-based and diffusion-based methods under planner guidance, InteractPro overcomes these challenges, ensuring richly motion-aware compositions. Extensive quantitative and qualitative evaluations demonstrate InteractPro's effectiveness in producing controllable, and coherent compositions across varied scenarios.