TY - JOUR
T1 - Meta-Reinforcement Learning for Reliable Communication in THz/VLC Wireless VR Networks
AU - Wang, Yining
AU - Chen, Mingzhe
AU - Yang, Zhaohui
AU - Saad, Walid
AU - Luo, Tao
AU - Cui, Shuguang
AU - Poor, H. Vincent
N1 - Funding Information:
This work was supported in part by the National Natural Science Foundation of China under Grant 62171047, in part by the U.S. National Science Foundation under Grant CNS- 1909372, in part by the National Key Research and Development Program of China under Grant 2018YFB1800800, in part by the Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone under Basic Research Project HZQB-KCZYZ-2021067, in part by the Shenzhen Outstanding Talents Training Fund 202002, in part by the Guangdong Research Projects (2017ZT07X152 and 2019CX01X104).
Publisher Copyright:
© 2022 IEEE.
PY - 2022/9/1
Y1 - 2022/9/1
N2 - In this paper, the problem of enhancing the quality of virtual reality (VR) services is studied for an indoor terahertz (THz)/visible light communication (VLC) wireless network. In the studied model, small base stations (SBSs) transmit high-quality VR images to VR users over THz bands and light-emitting diodes (LEDs) provide accurate indoor positioning services for them using VLC. Here, VR users move in real time and their movement patterns change over time according to their applications, where both THz and VLC links can be blocked by the bodies of VR users. To control the energy consumption of the studied THz/VLC wireless VR network, VLC access points (VAPs) must be selectively turned on so as to ensure accurate and extensive positioning for VR users. Based on the user positions, each SBS must generate corresponding VR images and establish THz links without body blockage to transmit the VR content. The problem is formulated as an optimization problem whose goal is to maximize the average number of successfully served VR users by selecting the appropriate VAPs to be turned on and controlling the user association with SBSs. To solve this problem, a policy gradient-based reinforcement learning (RL) algorithm that adopts a meta-learning approach is proposed. The proposed meta policy gradient (MPG) algorithm enables the trained policy to quickly adapt to new user movement patterns. In order to solve the problem of maximizing the average number of successfully served users for VR scenarios with large numbers of users, a low-complexity dual method based MPG algorithm (D-MPG) with a low complexity is proposed. Simulation results demonstrate that, compared to a baseline trust region policy optimization algorithm (TRPO), the proposed MPG and D-MPG algorithms yield up to 26.8% and 21.9% improvement in the average number of successfully served users as well as 81.2% and 87.5% gains in the convergence speed, respectively.
AB - In this paper, the problem of enhancing the quality of virtual reality (VR) services is studied for an indoor terahertz (THz)/visible light communication (VLC) wireless network. In the studied model, small base stations (SBSs) transmit high-quality VR images to VR users over THz bands and light-emitting diodes (LEDs) provide accurate indoor positioning services for them using VLC. Here, VR users move in real time and their movement patterns change over time according to their applications, where both THz and VLC links can be blocked by the bodies of VR users. To control the energy consumption of the studied THz/VLC wireless VR network, VLC access points (VAPs) must be selectively turned on so as to ensure accurate and extensive positioning for VR users. Based on the user positions, each SBS must generate corresponding VR images and establish THz links without body blockage to transmit the VR content. The problem is formulated as an optimization problem whose goal is to maximize the average number of successfully served VR users by selecting the appropriate VAPs to be turned on and controlling the user association with SBSs. To solve this problem, a policy gradient-based reinforcement learning (RL) algorithm that adopts a meta-learning approach is proposed. The proposed meta policy gradient (MPG) algorithm enables the trained policy to quickly adapt to new user movement patterns. In order to solve the problem of maximizing the average number of successfully served users for VR scenarios with large numbers of users, a low-complexity dual method based MPG algorithm (D-MPG) with a low complexity is proposed. Simulation results demonstrate that, compared to a baseline trust region policy optimization algorithm (TRPO), the proposed MPG and D-MPG algorithms yield up to 26.8% and 21.9% improvement in the average number of successfully served users as well as 81.2% and 87.5% gains in the convergence speed, respectively.
KW - Wireless virtual reality
KW - indoor positioning
KW - meta-learning
KW - reinforcement learning (RL)
KW - reliability
KW - terahertz (THz)
KW - visible light communications (VLC)
UR - http://www.scopus.com/inward/record.url?scp=85127475997&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85127475997&partnerID=8YFLogxK
U2 - 10.1109/TWC.2022.3161970
DO - 10.1109/TWC.2022.3161970
M3 - Article
AN - SCOPUS:85127475997
SN - 1536-1276
VL - 21
SP - 7778
EP - 7793
JO - IEEE Transactions on Wireless Communications
JF - IEEE Transactions on Wireless Communications
IS - 9
ER -