Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision

TL;DR

This paper presents an unsupervised approach using a maximum mean discrepancy variational autoencoder to learn representations of endoscopic videos, enabling tool presence detection with accuracy comparable to supervised methods.

Contribution

It introduces a novel unsupervised method for detecting surgical tools in endoscopic videos by manipulating learned latent representations without labeled data.

Findings

Achieved average precision of 71.56-76.18% in tool detection

Method performs comparably to supervised approaches

Demonstrates effectiveness of unsupervised representation learning for medical video analysis

Abstract

In this work, we explore whether it is possible to learn representations of endoscopic video frames to perform tasks such as identifying surgical tool presence without supervision. We use a maximum mean discrepancy (MMD) variational autoencoder (VAE) to learn low-dimensional latent representations of endoscopic videos and manipulate these representations to distinguish frames containing tools from those without tools. We use three different methods to manipulate these latent representations in order to predict tool presence in each frame. Our fully unsupervised methods can identify whether endoscopic video frames contain tools with average precision of 71.56, 73.93, and 76.18, respectively, comparable to supervised methods. Our code is available at https://github.com/zdavidli/tool-presence/