Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic

Xiongwei Wu, Xiuhua Li, Jun Li, P. C. Ching, Victor C.M. Leung, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

Abstract

Edge nodes (ENs) in Internet of Things commonly serve as gateways to cache sensing data while providing accessing services for data consumers. This paper considers multiple ENs that cache sensing data under the coordination of the cloud. Particularly, each EN can fetch content generated by sensors within its coverage, which can be uploaded to the cloud via fronthaul and then be delivered to other ENs beyond the communication range. However, sensing data are usually transient with time whereas frequent cache updates could lead to considerable energy consumption at sensors and fronthaul traffic loads. Therefore, we adopt Age of Information to evaluate data freshness and investigate intelligent caching policies to preserve data freshness while reducing cache update costs. Specifically, we model the cache update problem as a cooperative multi-agent Markov decision process with the goal of minimizing the long-term average weighted cost. To efficiently handle the exponentially large number of actions, we devise a novel reinforcement learning approach, which is a discrete multi-agent variant of soft actor-critic (SAC). Furthermore, we generalize the proposed approach into a decentralized control, where each EN can make decisions based on local observations only. Simulation results demonstrate the superior performance of the proposed SAC-based caching schemes.

Original languageEnglish (US)
JournalIEEE Transactions on Communications
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Keywords

  • age of information
  • cooperative multi-agent Markov decision process
  • Energy consumption
  • Intelligent sensors
  • Internet of Things
  • Internet of Things
  • Sensor phenomena and characterization
  • Sensors
  • soft actor-critic
  • Transient analysis
  • Wireless sensor networks

Fingerprint

Dive into the research topics of 'Caching Transient Content for IoT Sensing: Multi-Agent Soft Actor-Critic'. Together they form a unique fingerprint.

Cite this