Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

Xiongwei Wu, Xiuhua Li, Jun Li, P. C. Ching, Victor C.M. Leung, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Edge nodes (ENs) in Internet of Things commonly serve as gateways to cache sensing data while providing accessing services for data consumers. This paper considers multiple ENs that cache sensing data under the coordination of the cloud. Particularly, each EN can fetch content generated by sensors within its coverage, which can be uploaded to the cloud via fronthaul and then be delivered to other ENs beyond the communication range. However, sensing data are usually transient with time whereas frequent cache updates could lead to considerable energy consumption at sensors and fronthaul traffic loads. Therefore, we adopt Age of Information to evaluate data freshness and investigate intelligent caching policies to preserve data freshness while reducing cache update costs. Specifically, we model the cache update problem as a cooperative multi-agent Markov decision process with the goal of minimizing the long-term average weighted cost. To efficiently handle the exponentially large number of actions, we devise a novel reinforcement learning approach, which is a discrete multi-agent variant of soft actor-critic (SAC). Furthermore, we generalize the proposed approach into a decentralized control, where each EN can make decisions based on local observations only. Simulation results demonstrate the superior performance of the proposed SAC-based caching schemes.

Original languageEnglish (US)
Pages (from-to)5886-5901
Number of pages16
JournalIEEE Transactions on Communications
Volume69
Issue number9
DOIs
StatePublished - Sep 2021

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Keywords

  • Internet of things
  • age of information
  • cooperative multi-agent Markov decision process
  • soft actor-critic

Fingerprint

Dive into the research topics of 'Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic'. Together they form a unique fingerprint.

Cite this