Multi-Agent Reinforcement Learning for Cooperative Coded Caching via Homotopy Optimization

Xiongwei Wu, Jun Li, Ming Xiao, P. C. Ching, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

Abstract

Introducing cooperative coded caching into small cell networks is a promising approach to reducing traffic loads. By encoding content via maximum distance separable (MDS) codes, coded fragments can be collectively cached at small-cell base stations (SBSs) to enhance caching efficiency. However, content popularity is usually time-varying and unknown in practice. As a result, caching contents are anticipated to be intelligently updated by taking into account limited caching storage and interactive impacts among SBSs. In response to these challenges, we propose a multi-agent deep reinforcement learning (DRL) framework to intelligently update caching contents in dynamic environments. With the goal of minimizing long-term expected fronthaul traffic loads, we first model dynamic coded caching as a cooperative multi-agent Markov decision process. Owing to MDS coding, the resulting decision-making falls into a class of constrained reinforcement learning problems with continuous decision variables. To deal with this difficulty, we custom-build a novel DRL algorithm by embedding homotopy optimization into a deep deterministic policy gradient formalism. Next, to empower the caching framework with an effective trade-off between complexity and performance, we propose centralized, partially and fully decentralized caching controls by applying the derived DRL approach. Simulation results demonstrate the superior performance of the proposed multi-agent framework.

Original languageEnglish (US)
JournalIEEE Transactions on Wireless Communications
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics

Keywords

  • Centralized control
  • Complexity theory
  • Decision making
  • Encoding
  • MDS codes
  • Optimization
  • Reinforcement learning
  • Small cell networks
  • Wireless networks
  • deep multi-agent reinforcement learning
  • homotopy optimization

Fingerprint Dive into the research topics of 'Multi-Agent Reinforcement Learning for Cooperative Coded Caching via Homotopy Optimization'. Together they form a unique fingerprint.

Cite this