Self-organization in small cell networks: A reinforcement learning approach

Mehdi Bennis, Samir M. Perlaza, Pol Blasco, Zhu Han, H. Vincent Poor

Research output: Contribution to journalArticlepeer-review

165 Scopus citations


In this paper, a decentralized and self-organizing mechanism for small cell networks (such as micro-, femto-and picocells) is proposed. In particular, an application to the case in which small cell networks aim to mitigate the interference caused to the macrocell network, while maximizing their own spectral efficiencies, is presented. The proposed mechanism is based on new notions of reinforcement learning (RL) through which small cells jointly estimate their time-average performance and optimize their probability distributions with which they judiciously choose their transmit configurations. Here, a minimum signal to interference plus noise ratio (SINR) is guaranteed at the macrocell user equipment (UE), while the small cells maximize their individual performances. The proposed RL procedure is fully distributed as every small cell base station requires only an observation of its instantaneous performance which can be obtained from its UE. Furthermore, it is shown that the proposed mechanism always converges to an epsilon Nash equilibrium when all small cells share the same interest. In addition, this mechanism is shown to possess better convergence properties and incur less overhead than existing techniques such as best response dynamics, fictitious play or classical RL. Finally, numerical results are given to validate the theoretical findings, highlighting the inherent tradeoffs facing small cells, namely exploration/exploitation, myopic/foresighted behavior and complete/incomplete information.

Original languageEnglish (US)
Article number6542770
Pages (from-to)3202-3212
Number of pages11
JournalIEEE Transactions on Wireless Communications
Issue number7
StatePublished - 2013

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Applied Mathematics


  • Small cell networks
  • game theory
  • reinforcement learning
  • self-organizing networks


Dive into the research topics of 'Self-organization in small cell networks: A reinforcement learning approach'. Together they form a unique fingerprint.

Cite this