AdaGaR: Adaptive Gabor Representation for Dynamic Scene Reconstruction

TL;DR

AdaGaR introduces an adaptive Gabor-based framework with temporal regularization for improved dynamic scene reconstruction from monocular videos, achieving state-of-the-art results and robust generalization.

Contribution

It presents a novel adaptive Gabor representation and temporal continuity constraints, enhancing detail capture and motion stability in dynamic scene modeling.

Findings

Achieved state-of-the-art PSNR, SSIM, LPIPS on Tap-Vid DAVIS.

Demonstrated strong generalization in frame interpolation and stereo synthesis.

Enhanced stability and detail in dynamic scene reconstructions.

Abstract

Reconstructing dynamic 3D scenes from monocular videos requires simultaneously capturing high-frequency appearance details and temporally continuous motion. Existing methods using single Gaussian primitives are limited by their low-pass filtering nature, while standard Gabor functions introduce energy instability. Moreover, lack of temporal continuity constraints often leads to motion artifacts during interpolation. We propose AdaGaR, a unified framework addressing both frequency adaptivity and temporal continuity in explicit dynamic scene modeling. We introduce Adaptive Gabor Representation, extending Gaussians through learnable frequency weights and adaptive energy compensation to balance detail capture and stability. For temporal continuity, we employ Cubic Hermite Splines with Temporal Curvature Regularization to ensure smooth motion evolution. An Adaptive Initialization mechanism combining depth estimation, point tracking, and foreground masks establishes stable point cloud distributions in early training. Experiments on Tap-Vid DAVIS demonstrate state-of-the-art performance (PSNR 35.49, SSIM 0.9433, LPIPS 0.0723) and strong generalization across frame interpolation, depth consistency, video editing, and stereo view synthesis. Project page: https://jiewenchan.github.io/AdaGaR/