Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

Hao Zhou, Melike Erol-Kantarci, Vincent Poor

Research output: Contribution to journalArticlepeer-review

Abstract

Network slicing is a critical technique for 5G communications that covers radio access network (RAN), edge, transport and core slicing. The evolving network architecture requires the orchestration of multiple network resources such as radio and cache resources. In recent years, machine learning (ML) techniques have been widely applied for network management. However, most existing works do not take advantage of the knowledge transfer capability in ML. In this paper, we propose a deep transfer reinforcement learning (DTRL) scheme for joint radio and cache resource allocation to serve 5G RAN slicing. We first define a hierarchical architecture for joint resource allocation. Then we propose two DTRL algorithms: Q-value-based deep transfer reinforcement learning (QDTRL) and action selection-based deep transfer reinforcement learning (ADTRL). In the proposed schemes, learner agents utilize expert agents’ knowledge to improve their performance on current tasks. The proposed algorithms are compared with both the model-free exploration bonus deep Q-learning (EB-DQN) and the model-based priority proportional fairness and time-to-live (PPF-TTL) algorithms. Compared with EB-DQN, our proposed DTRL-based method presents 21.4% lower delay for Ultra Reliable Low Latency Communications (URLLC) slice and 22.4% higher throughput for enhanced Mobile Broad Band (eMBB) slice, while achieving significantly faster convergence than EB-DQN. Moreover, 40.8% lower URLLC delay and 59.8% higher eMBB throughput are observed with respect to PPF-TTL.

Original languageEnglish (US)
Pages (from-to)1
Number of pages1
JournalIEEE Transactions on Cognitive Communications and Networking
DOIs
StateAccepted/In press - 2022

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Keywords

  • 5G
  • 5G mobile communication
  • Convergence
  • Knowledge transfer
  • Resource management
  • Task analysis
  • Training
  • Ultra reliable low latency communication
  • edge caching
  • network slicing
  • transfer reinforcement learning

Fingerprint

Dive into the research topics of 'Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing'. Together they form a unique fingerprint.

Cite this