One-shot skill assessment in high-stakes domains with limited data via meta learning

TL;DR

This paper presents A-VBANet, a meta-learning model enabling domain-agnostic skill assessment with minimal data, achieving high accuracy in simulated and real surgical tasks, thus broadening DL applicability in high-stakes fields.

Contribution

Introduction of A-VBANet, a novel meta-learning approach for one-shot skill assessment that generalizes across domains with limited data, a first in critical skill evaluation.

Findings

Achieved up to 99.5% accuracy in simulated tasks

Achieved 89.7% accuracy in real laparoscopic surgery

Demonstrated effective domain adaptation with minimal data

Abstract

Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5% in one-shot and 99.9% in few-shot settings for simulated tasks and 89.7% for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data.