MetaSketch: Wireless Semantic Segmentation by Reconfigurable Intelligent Surfaces

Jingzhi Hu, Hongliang Zhang, Kaigui Bian, Zhu Han, H. Vincent Poor, Lingyang Song

Research output: Contribution to journalArticlepeer-review

8 Scopus citations


Semantic segmentation is a process of partitioning an image into segments for recognizing regions of humans and objects, which can be widely applied in scenarios such as healthcare and safety monitoring. To avoid privacy violation, using radio frequency (RF) signals instead of photos for semantic segmentation has gained increasing attention. However, traditional human and object recognition by using RF signals is a passive signal collection and analysis process without changing the radio environment. The recognition accuracy is restricted significantly by unwanted multi-path fading, and/or the limited number of independent channels between RF transceivers. This paper introduces MetaSketch, a novel RF-sensing system that performs semantic recognition and segmentation for humans and objects by making the radio environment reconfigurable. A metamaterial-based reconfigurable intelligent surface is incorporated to diversify the information carried by RF signals. Using compressive sensing techniques, MetaSketch reconstructs a point cloud consisting of the reflection coefficients of humans and objects at different spatial points, and recognizes the semantic meaning of the points by using symmetric multilayer perceptron groups. Our evaluation results show that MetaSketch is capable of generating favorable radio environments, extracting exact point clouds, and labeling the semantic meaning of the points with an average error rate of less than 1% in an indoor space.

Original languageEnglish (US)
Pages (from-to)5916-5929
Number of pages14
JournalIEEE Transactions on Wireless Communications
Issue number8
StatePublished - Aug 1 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


  • RF sensing
  • compressive sensing
  • reconfigurable intelligent surface
  • semantic segmentation


Dive into the research topics of 'MetaSketch: Wireless Semantic Segmentation by Reconfigurable Intelligent Surfaces'. Together they form a unique fingerprint.

Cite this