Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

Xiongwei Wu, Xiuhua Li, Jun Li, P. C. Ching, Victor C.M. Leung, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

28 Scopus citations


Edge nodes (ENs) in Internet of Things commonly serve as gateways to cache sensing data while providing accessing services for data consumers. This paper considers multiple ENs that cache sensing data under the coordination of the cloud. Particularly, each EN can fetch content generated by sensors within its coverage, which can be uploaded to the cloud via fronthaul and then be delivered to other ENs beyond the communication range. However, sensing data are usually transient with time whereas frequent cache updates could lead to considerable energy consumption at sensors and fronthaul traffic loads. Therefore, we adopt Age of Information to evaluate data freshness and investigate intelligent caching policies to preserve data freshness while reducing cache update costs. Specifically, we model the cache update problem as a cooperative multi-agent Markov decision process with the goal of minimizing the long-term average weighted cost. To efficiently handle the exponentially large number of actions, we devise a novel reinforcement learning approach, which is a discrete multi-agent variant of soft actor-critic (SAC). Furthermore, we generalize the proposed approach into a decentralized control, where each EN can make decisions based on local observations only. Simulation results demonstrate the superior performance of the proposed SAC-based caching schemes.

Original languageEnglish (US)
Pages (from-to)5886-5901
Number of pages16
JournalIEEE Transactions on Communications
Issue number9
StatePublished - Sep 2021
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering


  • Internet of things
  • age of information
  • cooperative multi-agent Markov decision process
  • soft actor-critic


Dive into the research topics of 'Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic'. Together they form a unique fingerprint.

Cite this