MeshOn: Intersection-Free Mesh-to-Mesh Composition

TL;DR

MeshOn is a novel method for realistic, intersection-free mesh-to-mesh composition that combines vision-language alignment, geometric optimization, and physics-inspired constraints.

Contribution

It introduces a multi-step optimization framework that integrates vision-language models, geometric losses, and physics-inspired barriers for mesh composition.

Findings

Successfully fits accessories onto target meshes across various materials.

Outperforms traditional registration algorithms and generative approaches.

Robustly maintains intersection-free, realistic compositions.

Abstract

We propose MeshOn, a method that finds physically and semantically realistic compositions of two input meshes. Given an accessory, a base mesh with a user-defined target region, and optional text strings for both meshes, MeshOn uses a multi-step optimization framework to realistically fit the meshes onto each other while preventing intersections. We initialize the shapes' rigid configuration via a structured alignment scheme using Vision-to-Language Models, which we then optimize using a combination of attractive geometric losses, and a physics-inspired barrier loss that prevents surface intersections. We then obtain a final deformation of the object, assisted by a diffusion prior. Our method successfully fits accessories of various materials over a breadth of target regions, and is designed to fit directly into existing digital artist workflows. We demonstrate the robustness and accuracy of our pipeline by comparing it with generative approaches and traditional registration algorithms.