XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding

TL;DR

XSpecMesh introduces a multi-head speculative decoding approach with verification and distillation to accelerate auto-regressive mesh generation, achieving 1.7x faster inference while maintaining high quality.

Contribution

The paper presents a novel multi-head speculative decoding scheme with verification and distillation strategies for auto-regressive mesh generation models.

Findings

Achieves 1.7x inference speedup

Maintains high mesh quality

Effective in accelerating mesh generation

Abstract

Current auto-regressive models can generate high-quality, topologically precise meshes; however, they necessitate thousands-or even tens of thousands-of next-token predictions during inference, resulting in substantial latency. We introduce XSpecMesh, a quality-preserving acceleration method for auto-regressive mesh generation models. XSpecMesh employs a lightweight, multi-head speculative decoding scheme to predict multiple tokens in parallel within a single forward pass, thereby accelerating inference. We further propose a verification and resampling strategy: the backbone model verifies each predicted token and resamples any tokens that do not meet the quality criteria. In addition, we propose a distillation strategy that trains the lightweight decoding heads by distilling from the backbone model, encouraging their prediction distributions to align and improving the success rate of speculative predictions. Extensive experiments demonstrate that our method achieves a 1.7x speedup without sacrificing generation quality. Our code will be released.