Stochastic power adaptation with multi-agent reinforcement learning for cognitive wireless mesh networks

Xianfu Chen, Z. Zhao, H. Zhang

    Research output: Contribution to journalArticleScientificpeer-review

    43 Citations (Scopus)

    Abstract

    As the scarce spectrum resource is becoming overcrowded, cognitive radio indicates great flexibility to improve the spectrum efficiency by opportunistically accessing the authorized frequency bands. One of the critical challenges for operating such radios in a network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of the primary users. In this paper, we focus on the noncooperative power allocation problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of energy efficiency. Due to the SUs' dynamic and spontaneous properties, the problem is modeled as a stochastic learning process. We first extend the single-agent Q-learning to a multiuser context, and then propose a conjecture-based multiagent Q-learning algorithm to achieve the optimal transmission strategies with only private and incomplete information. An intelligent SU performs Q-function updates based on the conjecture over the other SUs' stochastic behaviors. This learning algorithm provably converges given certain restrictions that arise during the learning procedure. Simulation experiments are used to verify the performance of our algorithm and demonstrate its effectiveness of improving the energy efficiency.
    Original languageEnglish
    Pages (from-to)2155-2166
    Number of pages12
    JournalIEEE Transactions on Mobile Computing
    Volume12
    Issue number11
    DOIs
    Publication statusPublished - 2013
    MoE publication typeA1 Journal article-refereed

      Fingerprint

    Keywords

    • Communication/Networking and Information Technology
    • Computer Systems Organization
    • distributed networks
    • energy-aware systems
    • hardware
    • network architecture and design
    • power management

    Cite this