ARGS: Auto-Regressive Gaussian Splatting via Parallel Progressive Next-Scale Prediction

TL;DR

This paper introduces ARGS, a novel auto-regressive framework for 3D object generation using Gaussian splatting, enabling efficient multi-scale detail prediction with hierarchical trees and transformer-based structure modeling.

Contribution

The paper presents a new parallel next-scale prediction method for 3D generation, combining Gaussian simplification, hierarchical trees, and transformers for improved efficiency and structural consistency.

Findings

Efficient generation with alog n steps using hierarchical trees.

Effective multi-scale Gaussian representations with controllable detail.

High visual fidelity and manageable computation time.

Abstract

Auto-regressive frameworks for next-scale prediction of 2D images have demonstrated strong potential for producing diverse and sophisticated content by progressively refining a coarse input. However, extending this paradigm to 3D object generation remains largely unexplored. In this paper, we introduce auto-regressive Gaussian splatting (ARGS), a framework for making next-scale predictions in parallel for generation according to levels of detail. We propose a Gaussian simplification strategy and reverse the simplification to guide next-scale generation. Benefiting from the use of hierarchical trees, the generation process requires only \(\mathcal{O}(\log n)\) steps, where \(n\) is the number of points. Furthermore, we propose a tree-based transformer to predict the tree structure auto-regressively, allowing leaf nodes to attend to their internal ancestors to enhance structural consistency. Extensive experiments demonstrate that our approach effectively generates multi-scale Gaussian representations with controllable levels of detail, visual fidelity, and a manageable time consumption budget.