Rethinking Point Clouds as Sequences: A Causal Next-Token Predictive Learning Framework

TL;DR

This paper introduces PointNTP, a novel causal, decoder-free pre-training framework for 3D point clouds that models structural dependencies directly in latent space using a Transformer-based sequence prediction approach.

Contribution

It reformulates point cloud pre-training as a causal next-token prediction task, moving away from reconstruction-based methods and enabling scalable, modality-agnostic learning.

Findings

Achieves state-of-the-art results on multiple 3D classification and segmentation benchmarks.

Demonstrates the effectiveness of causal latent prediction over traditional reconstruction methods.

Provides a simple, scalable paradigm for self-supervised learning in 3D point clouds.

Abstract

With the rapid progress of multimodal foundation models and predictive pre-training, an important open question is how to equip 3D point clouds with a pre-training paradigm that is better aligned with next-token and next-embedding learning. Existing point-cloud self-supervised methods are largely built on masked reconstruction or explicit geometric generation, and thus remain tied to input recovery rather than predictive dependency modeling. In this paper, we introduce PointNTP, which reformulates point cloud pre-training as a fully causal, decoder-free latent Next-Token Prediction problem. Specifically, each point cloud is first partitioned into local patches and serialized into a structured 3D token sequence according to patch-center geometry. The resulting sequence is then modeled by a causal Transformer under prefix-only conditioning, and trained with a shift-based prediction objective stabilized by stop-gradient targets. This design enables the model to learn structural dependencies directly in latent space, without reconstruction decoders or explicit geometric recovery. Extensive experiments demonstrate that the proposed PointNTP is highly competitive across multiple downstream tasks: it achieves 93.8%(+0.5%), 92.6%(+0.3%), and 89.3%(+1.1%) on OBJ_BG, OBJ_ONLY, and PB_T50_RS of ScanObjectNN, respectively; obtains 85.0%(+0.1%) in Cls.mIoU on ShapeNetPart; and reaches 71.1% mAcc on S3DIS Area 5. Overall, decoder-free causal latent prediction provides a simple, scalable, and potentially modality-agnostic paradigm for point-cloud self-supervised learning, offering a new 3D perspective on foundation-style predictive learning for 3D data.