Invisible-to-Visible: Privacy-Aware Human Segmentation using Airborne Ultrasound via Collaborative Learning Probabilistic U-Net

TL;DR

This paper introduces a privacy-preserving human segmentation method using airborne ultrasound and a collaborative probabilistic U-Net, enabling accurate segmentation solely from ultrasound data.

Contribution

It proposes a novel collaborative learning framework that aligns ultrasound and visual segmentation latent spaces, improving segmentation accuracy without visual data during inference.

Findings

Outperforms conventional probabilistic U-Net in segmentation accuracy

Effectively estimates human shapes using only ultrasound data

Demonstrates potential for privacy-aware indoor human detection

Abstract

Color images are easy to understand visually and can acquire a great deal of information, such as color and texture. They are highly and widely used in tasks such as segmentation. On the other hand, in indoor person segmentation, it is necessary to collect person data considering privacy. We propose a new task for human segmentation from invisible information, especially airborne ultrasound. We first convert ultrasound waves to reflected ultrasound directional images (ultrasound images) to perform segmentation from invisible information. Although ultrasound images can roughly identify a person's location, the detailed shape is ambiguous. To address this problem, we propose a collaborative learning probabilistic U-Net that uses ultrasound and segmentation images simultaneously during training, closing the probabilistic distributions between ultrasound and segmentation images by comparing the parameters of the latent spaces. In inference, only ultrasound images can be used to obtain segmentation results. As a result of performance verification, the proposed method could estimate human segmentations more accurately than conventional probabilistic U-Net and other variational autoencoder models.