VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Image

TL;DR

VecSet-Edit introduces a novel mesh editing pipeline leveraging pre-trained VecSet LRM, enabling precise, high-fidelity editing from single images while preserving geometric and textural details.

Contribution

It is the first to utilize VecSet LRM for mesh editing, proposing new strategies for localizing regions and rejecting outliers during denoising.

Findings

Effective local region editing guided by 2D images

Preserves geometric and textural details of meshes

Outperforms voxel-based methods in resolution and efficiency

Abstract

3D editing has emerged as a critical research area to provide users with flexible control over 3D assets. While current editing approaches predominantly focus on 3D Gaussian Splatting or multi-view images, the direct editing of 3D meshes remains underexplored. Prior attempts, such as VoxHammer, rely on voxel-based representations that suffer from limited resolution and necessitate labor-intensive 3D mask. To address these limitations, we propose \textbf{VecSet-Edit}, the first pipeline that leverages the high-fidelity VecSet Large Reconstruction Model (LRM) as a backbone for mesh editing. Our approach is grounded on a analysis of the spatial properties in VecSet tokens, revealing that token subsets govern distinct geometric regions. Based on this insight, we introduce Mask-guided Token Seeding and Attention-aligned Token Gating strategies to precisely localize target regions using only 2D image conditions. Also, considering the difference between VecSet diffusion process versus voxel we design a Drift-aware Token Pruning to reject geometric outliers during the denoising process. Finally, our Detail-preserving Texture Baking module ensures that we not only preserve the geometric details of original mesh but also the textural information. More details can be found in our project page: https://github.com/BlueDyee/VecSet-Edit/tree/main