Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

Xiangnan Liu, Haijun Zhang, Keping Long, Mingyu Zhou, Yonghui Li, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, an IRS-Aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-Threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-Threading version in terms of transmit beamforming and phase-shift design.

Original languageEnglish (US)
Pages (from-to)2056-2069
Number of pages14
JournalIEEE Journal on Selected Areas in Communications
Volume40
Issue number7
DOIs
StatePublished - Jul 1 2022

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Electrical and Electronic Engineering

Keywords

  • Integrated sensing and communications
  • distributed reinforcement learning
  • intelligent reflecting surface
  • phase shift design
  • transmit beamforming

Fingerprint

Dive into the research topics of 'Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band'. Together they form a unique fingerprint.

Cite this