Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

Xiangnan Liu; Haijun Zhang; Keping Long; Mingyu Zhou; Yonghui Li; H. Vincent Poor

doi:10.1109/JSAC.2022.3158696

Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

Xiangnan Liu, Haijun Zhang, Keping Long, Mingyu Zhou, Yonghui Li, H. Vincent Poor

Research output: Contribution to journal › Article › peer-review

44 Scopus citations

Abstract

In this paper, an IRS-Aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-Threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-Threading version in terms of transmit beamforming and phase-shift design.

Original language	English (US)
Pages (from-to)	2056-2069
Number of pages	14
Journal	IEEE Journal on Selected Areas in Communications
Volume	40
Issue number	7
DOIs	https://doi.org/10.1109/JSAC.2022.3158696
State	Published - Jul 1 2022

All Science Journal Classification (ASJC) codes

Computer Networks and Communications
Electrical and Electronic Engineering

Keywords

Integrated sensing and communications
distributed reinforcement learning
intelligent reflecting surface
phase shift design
transmit beamforming

Access to Document

10.1109/JSAC.2022.3158696

Cite this

@article{ea2f32fd16a449e1b118175582ee85f5,

title = "Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band",

abstract = "In this paper, an IRS-Aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-Threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-Threading version in terms of transmit beamforming and phase-shift design.",

keywords = "Integrated sensing and communications, distributed reinforcement learning, intelligent reflecting surface, phase shift design, transmit beamforming",

author = "Xiangnan Liu and Haijun Zhang and Keping Long and Mingyu Zhou and Yonghui Li and Poor, {H. Vincent}",

note = "Publisher Copyright: {\textcopyright} 1983-2012 IEEE.",

year = "2022",

month = jul,

day = "1",

doi = "10.1109/JSAC.2022.3158696",

language = "English (US)",

volume = "40",

pages = "2056--2069",

journal = "IEEE Journal on Selected Areas in Communications",

issn = "0733-8716",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "7",

}

TY - JOUR

T1 - Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

AU - Liu, Xiangnan

AU - Zhang, Haijun

AU - Long, Keping

AU - Zhou, Mingyu

AU - Li, Yonghui

AU - Poor, H. Vincent

PY - 2022/7/1

Y1 - 2022/7/1

N2 - In this paper, an IRS-Aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-Threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-Threading version in terms of transmit beamforming and phase-shift design.

AB - In this paper, an IRS-Aided integrated sensing and communications (ISAC) system operating in the terahertz (THz) band is proposed to maximize the system capacity. Transmit beamforming and phase-shift design are transformed into a universal optimization problem with ergodic constraints. Then the joint optimization of transmit beamforming and phase-shift design is achieved by gradient-based, primal-dual proximal policy optimization (PPO) in the multi-user multiple-input single-output (MISO) scenario. Specifically, the actor part generates continuous transmit beamforming and the critic part takes charge of discrete phase shift design. Based on the MISO scenario, we investigate a distributed PPO (DPPO) framework with the concept of multi-Threading learning in the multi-user multiple-input multiple-output (MIMO) scenario. Simulation results demonstrate the effectiveness of the primal-dual PPO algorithm and its multi-Threading version in terms of transmit beamforming and phase-shift design.

KW - Integrated sensing and communications

KW - distributed reinforcement learning

KW - intelligent reflecting surface

KW - phase shift design

KW - transmit beamforming

UR - http://www.scopus.com/inward/record.url?scp=85126281531&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85126281531&partnerID=8YFLogxK

U2 - 10.1109/JSAC.2022.3158696

DO - 10.1109/JSAC.2022.3158696

M3 - Article

AN - SCOPUS:85126281531

SN - 0733-8716

VL - 40

SP - 2056

EP - 2069

JO - IEEE Journal on Selected Areas in Communications

JF - IEEE Journal on Selected Areas in Communications

IS - 7

ER -

Proximal Policy Optimization-Based Transmit Beamforming and Phase-Shift Design in an IRS-Aided ISAC System for the THz Band

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this