Mesh-Pro: Asynchronous Advantage-guided Ranking Preference Optimization for Artist-style Quadrilateral Mesh Generation

TL;DR

Mesh-Pro introduces an asynchronous RL framework with a novel preference optimization algorithm for efficient and high-quality 3D mesh generation, outperforming existing methods in speed and generalization.

Contribution

The paper presents the first asynchronous online RL framework for 3D mesh generation and proposes ARPO, a new preference optimization algorithm, enhancing efficiency and quality.

Findings

Mesh-Pro is 3.75× faster than synchronous RL.

ARPO outperforms DPO and GRPO in generalization.

Mesh-Pro achieves state-of-the-art results on artistic and dense meshes.

Abstract

Reinforcement learning (RL) has demonstrated remarkable success in text and image generation, yet its potential in 3D generation remains largely unexplored. Existing attempts typically rely on offline direct preference optimization (DPO) method, which suffers from low training efficiency and limited generalization. In this work, we aim to enhance both the training efficiency and generation quality of RL in 3D mesh generation. Specifically, (1) we design the first asynchronous online RL framework tailored for 3D mesh generation post-training efficiency improvement, which is 3.75$\times$ faster than synchronous RL. (2) We propose Advantage-guided Ranking Preference Optimization (ARPO), a novel RL algorithm that achieves a better trade-off between training efficiency and generalization than current RL algorithms designed for 3D mesh generation, such as DPO and group relative policy optimization (GRPO). (3) Based on asynchronous ARPO, we propose Mesh-Pro, which additionally introduces a novel diagonal-aware mixed triangular-quadrilateral tokenization for mesh representation and a ray-based reward for geometric integrity. Mesh-Pro achieves state-of-the-art performance on artistic and dense meshes.