Expanded Comprehensive Robotic Cholecystectomy Dataset (CRCD)

TL;DR

This paper introduces an expanded, comprehensive dataset for robotic cholecystectomy, including detailed kinematic, pedal, and visual data, to advance surgical robotics research and automation.

Contribution

It provides a unique, detailed dataset with annotations and surgeon diversity, addressing gaps in existing datasets for improved surgical automation and skill evaluation.

Findings

Models predicting clutch and camera usage demonstrate dataset utility.

3D scene reconstruction showcases potential for surgical automation.

Keypoint and segmentation models achieve promising results.

Abstract

In recent years, the application of machine learning to minimally invasive surgery (MIS) has attracted considerable interest. Datasets are critical to the use of such techniques. This paper presents a unique dataset recorded during ex vivo pseudo-cholecystectomy procedures on pig livers using the da Vinci Research Kit (dVRK). Unlike existing datasets, it addresses a critical gap by providing comprehensive kinematic data, recordings of all pedal inputs, and offers a time-stamped record of the endoscope's movements. This expanded version also includes segmentation and keypoint annotations of images, enhancing its utility for computer vision applications. Contributed by seven surgeons with varied backgrounds and experience levels that are provided as a part of this expanded version, the dataset is an important new resource for surgical robotics research. It enables the development of advanced methods for evaluating surgeon skills, tools for providing better context awareness, and automation of surgical tasks. Our work overcomes the limitations of incomplete recordings and imprecise kinematic data found in other datasets. To demonstrate the potential of the dataset for advancing automation in surgical robotics, we introduce two models that predict clutch usage and camera activation, a 3D scene reconstruction example, and the results from our keypoint and segmentation models.