# eNCApsulate: neural cellular automata for precision diagnosis on capsule endoscopes

## TL;DR

This paper introduces eNCApsulate, a system using neural cellular automata to enable real-time diagnosis and localization in wireless capsule endoscopes.

## Contribution

The first work to enable reliable bleeding segmentation and depth estimation on miniaturized devices like wireless capsule endoscopes.

## Key findings

- NCAs achieve higher accuracy than other portable models with over 100x fewer parameters.
- Depth estimation results from NCAs are visually convincing and sometimes exceed pseudo-ground truth detail.
- Runtime optimizations on the ESP32-S3 microcontroller significantly improve inference speed.

## Abstract

Wireless capsule endoscopy (WCE) is a noninvasive imaging method for the entire gastrointestinal tract and is a pain-free alternative to traditional endoscopy. It generates extensive video data that requires significant review time, and localizing the capsule after ingestion is a challenge. Techniques like bleeding detection and depth estimation can help with localization of pathologies, but deep learning models are typically too large to run directly on the capsule.

Neural cellular automata (NCAs) for bleeding segmentation and depth estimation are trained on capsule endoscopic images. For monocular depth estimation, we distill a large foundation model into the lean NCA architecture, by treating the outputs of the foundation model as pseudo-ground truth. We then port the trained NCAs to the ESP32 microcontroller, enabling efficient image processing on hardware as small as a camera capsule.

NCAs are more accurate (Dice) than other portable segmentation models, while requiring more than 100x fewer parameters stored in memory than other small-scale models. The visual results of NCAs depth estimation look convincing and in some cases beat the realism and detail of the pseudo-ground truth. Runtime optimizations on the ESP32-S3 accelerate the average inference speed significantly, by more than factor 3.

With several algorithmic adjustments and distillation, it is possible to eNCApsulate NCA models into microcontrollers that fit into wireless capsule endoscopes. This is the first work that enables reliable bleeding segmentation and depth estimation on a miniaturized device, paving the way for precise diagnosis combined with visual odometry as a means of precise localization of the capsule—on the capsule.

The online version contains supplementary material available at 10.1007/s11548-025-03425-x.