NRTR: Neuron Reconstruction with Transformer from 3D Optical Microscopy Images

TL;DR

This paper introduces NRTR, an end-to-end deep learning model based on Transformer architecture, for neuron reconstruction from 3D optical microscopy images, simplifying the process and improving accuracy.

Contribution

NRTR is the first image-to-set deep learning model for end-to-end neuron reconstruction, eliminating complex rule-based components.

Findings

NRTR outperforms existing methods on BigNeuron and VISoR-40 datasets.

NRTR demonstrates effective set-prediction approach for neuron reconstruction.

The model simplifies training and improves reconstruction accuracy.

Abstract

The neuron reconstruction from raw Optical Microscopy (OM) image stacks is the basis of neuroscience. Manual annotation and semi-automatic neuron tracing algorithms are time-consuming and inefficient. Existing deep learning neuron reconstruction methods, although demonstrating exemplary performance, greatly demand complex rule-based components. Therefore, a crucial challenge is designing an end-to-end neuron reconstruction method that makes the overall framework simpler and model training easier. We propose a Neuron Reconstruction Transformer (NRTR) that, discarding the complex rule-based components, views neuron reconstruction as a direct set-prediction problem. To the best of our knowledge, NRTR is the first image-to-set deep learning model for end-to-end neuron reconstruction. In experiments using the BigNeuron and VISoR-40 datasets, NRTR achieves excellent neuron reconstruction results for comprehensive benchmarks and outperforms competitive baselines. Results of extensive experiments indicate that NRTR is effective at showing that neuron reconstruction is viewed as a set-prediction problem, which makes end-to-end model training available.