Image-free single-pixel segmentation

TL;DR

This paper introduces an innovative image-free single-pixel segmentation method that uses optimized structured illumination and neural networks to achieve high-accuracy segmentation with significantly less data, suitable for resource-limited platforms.

Contribution

The paper presents a novel end-to-end framework combining optimized illumination patterns and neural network inference for efficient, high-accuracy segmentation without high-fidelity images.

Findings

Achieves over 80% Dice coefficient at 1% sampling ratio.

Reaches over 96% pixel accuracy with minimal input data.

Validated through both simulation and real experiments.

Abstract

The existing segmentation techniques require high-fidelity images as input to perform semantic segmentation. Since the segmentation results contain most of edge information that is much less than the acquired images, the throughput gap leads to both hardware and software waste. In this letter, we report an image-free single-pixel segmentation technique. The technique combines structured illumination and single-pixel detection together, to efficiently samples and multiplexes scene's segmentation information into compressed one-dimensional measurements. The illumination patterns are optimized together with the subsequent reconstruction neural network, which directly infers segmentation maps from the single-pixel measurements. The end-to-end encoding-and-decoding learning framework enables optimized illumination with corresponding network, which provides both high acquisition and segmentation efficiency. Both simulation and experimental results validate that accurate segmentation can be achieved using two-order-of-magnitude less input data. When the sampling ratio is 1%, the Dice coefficient reaches above 80% and the pixel accuracy reaches above 96%. We envision that this image-free segmentation technique can be widely applied in various resource-limited platforms such as UAV and unmanned vehicle that require real-time sensing.