Future Slot Prediction for Unsupervised Object Discovery in Surgical Video

TL;DR

This paper introduces a dynamic temporal slot transformer that improves unsupervised object discovery in surgical videos, enabling better interpretation of complex scenes for healthcare applications.

Contribution

It proposes a novel DTST module that enhances slot prediction over time, addressing challenges in parsing heterogeneous surgical scenes.

Findings

Achieves state-of-the-art results on surgical datasets

Effective in real-time surgical video interpretation

Improves unsupervised object discovery in complex scenes

Abstract

Object-centric slot attention is an emerging paradigm for unsupervised learning of structured, interpretable object-centric representations (slots). This enables effective reasoning about objects and events at a low computational cost and is thus applicable to critical healthcare applications, such as real-time interpretation of surgical video. The heterogeneous scenes in real-world applications like surgery are, however, difficult to parse into a meaningful set of slots. Current approaches with an adaptive slot count perform well on images, but their performance on surgical videos is low. To address this challenge, we propose a dynamic temporal slot transformer (DTST) module that is trained both for temporal reasoning and for predicting the optimal future slot initialization. The model achieves state-of-the-art performance on multiple surgical databases, demonstrating that unsupervised object-centric methods can be applied to real-world data and become part of the common arsenal in healthcare applications.