TOGS: Gaussian Splatting with Temporal Opacity Offset for Real-Time 4D DSA Rendering

TL;DR

TOGS introduces a novel Gaussian splatting technique with temporal opacity offset for 4D DSA, significantly enhancing rendering quality and speed in sparse view scenarios for medical imaging.

Contribution

The paper proposes TOGS, a Gaussian splatting method with opacity offset modeling temporal variations, improving 4D DSA rendering quality and efficiency over prior approaches.

Findings

Achieves state-of-the-art rendering quality with fewer views

Enables real-time 4D DSA visualization

Reduces storage overhead through Gaussian pruning

Abstract

Four-dimensional Digital Subtraction Angiography (4D DSA) is a medical imaging technique that provides a series of 2D images captured at different stages and angles during the process of contrast agent filling blood vessels. It plays a significant role in the diagnosis of cerebrovascular diseases. Improving the rendering quality and speed under sparse sampling is important for observing the status and location of lesions. The current methods exhibit inadequate rendering quality in sparse views and suffer from slow rendering speed. To overcome these limitations, we propose TOGS, a Gaussian splatting method with opacity offset over time, which can effectively improve the rendering quality and speed of 4D DSA. We introduce an opacity offset table for each Gaussian to model the opacity offsets of the Gaussian, using these opacity-varying Gaussians to model the temporal variations in the radiance of the contrast agent. By interpolating the opacity offset table, the opacity variation of the Gaussian at different time points can be determined. This enables us to render the 2D DSA image at that specific moment. Additionally, we introduced a Smooth loss term in the loss function to mitigate overfitting issues that may arise in the model when dealing with sparse view scenarios. During the training phase, we randomly prune Gaussians, thereby reducing the storage overhead of the model. The experimental results demonstrate that compared to previous methods, this model achieves state-of-the-art render quality under the same number of training views. Additionally, it enables real-time rendering while maintaining low storage overhead. The code is available at https://github.com/hustvl/TOGS.