Pianist Transformer: Towards Expressive Piano Performance Rendering via Scalable Self-Supervised Pre-Training

TL;DR

Pianist Transformer leverages self-supervised pre-training and a scalable architecture to significantly improve expressive piano performance rendering, achieving human-like quality without relying on small labeled datasets.

Contribution

It introduces a unified MIDI representation, an efficient asymmetric architecture, and a large-scale self-supervised pre-training pipeline for expressive music performance.

Findings

Achieves state-of-the-art objective metrics

Attains human-level subjective ratings

Enables longer context modeling and faster inference

Abstract

Existing methods for expressive music performance rendering rely on supervised learning over small labeled datasets, which limits scaling of both data volume and model size, despite the availability of vast unlabeled music, as in vision and language. To address this gap, we introduce Pianist Transformer, with four key contributions: 1) a unified Musical Instrument Digital Interface (MIDI) data representation for learning the shared principles of musical structure and expression without explicit annotation; 2) an efficient asymmetric architecture, enabling longer contexts and faster inference without sacrificing rendering quality; 3) a self-supervised pre-training pipeline with 10B tokens and 135M-parameter model, unlocking data and model scaling advantages for expressive performance rendering; 4) a state-of-the-art performance model, which achieves strong objective metrics and human-level subjective ratings. Overall, Pianist Transformer establishes a scalable path toward human-like performance synthesis in the music domain.