MetaSketch: Wireless Semantic Segmentation by Metamaterial Surfaces

TL;DR

MetaSketch is a novel RF-sensing system that uses reconfigurable metamaterial surfaces and compressive sensing to achieve high-accuracy semantic segmentation of humans and objects without visual data.

Contribution

It introduces a reconfigurable radio environment using metamaterials and applies compressive sensing for improved RF-based semantic segmentation.

Findings

Achieves less than 1% average error rate in indoor semantic segmentation

Effectively reconstructs point clouds of humans and objects from RF signals

Demonstrates the potential of reconfigurable radio environments for privacy-preserving recognition

Abstract

Semantic segmentation is a process of partitioning an image into multiple segments for recognizing humans and objects, which can be widely applied in scenarios such as healthcare and safety monitoring. To avoid privacy violation, using RF signals instead of an image for human and object recognition has gained increasing attention. However, human and object recognition by using RF signals is usually a passive signal collection and analysis process without changing the radio environment, and the recognition accuracy is restricted significantly by unwanted multi-path fading, and/or the limited number of independent channels between RF transceivers in uncontrollable radio environments. This paper introduces MetaSketch, a novel RF-sensing system that performs semantic recognition and segmentation for humans and objects by making the radio environment reconfigurable. A metamaterial surface is incorporated into MetaSketch and diversifies the information carried by RF signals. Using compressive sensing techniques, MetaSketch reconstructs a point cloud consisting of the reflection coefficients of humans and objects at different spatial points, and recognizes the semantic meaning of the points by using symmetric multilayer perceptron groups. Our evaluation results show that MetaSketch is capable of generating favorable radio environments and extracting exact point clouds, and labeling the semantic meaning of the points with an average error rate of less than 1% in an indoor space.