Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications

Xianfu Chen, Zhifeng Zhao, Honggang Zhang, Tao Chen

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    5 Citations (Scopus)


    Cognitive wireless mesh networks have great potential to green communication. One of the critical challenges for realizing such networks is how to adaptively match transmit powers and allocate frequency resources among secondary users of the licensed frequency bands whilst maintaining the quality-of-service (QoS) requirement of the primary users, even in mutually entangled interferences environment. In this paper, we discuss the power assignment matching problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of the Signal-to-Interference-plus-Noise Ratio (SINR) requirement of each secondary user (SU), the mean-squared error (MSE) constraint by the primary users, and the energy efficiency. Due to the secondary users' selfish and spontaneous features, the problem is modeled as a stochastic non-cooperative game. We extend the conventional single-agent Q-learning to a non-cooperative multi-agent learning context, using the framework of stochastic non-cooperative games. Within the multi-agent Q-learning processes, a learning SU maintains Q-functions over joint actions set, and performs updating based on the conjecture about the other SUs' behaviors over the current Q-values. Numerical experiments on a hybrid CogMesh consisting two SUs and one specific PU suggest validity and efficiency of the proposed algorithm.
    Original languageEnglish
    Title of host publication2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops)
    Subtitle of host publicationIstanbul, Turkey, 26-30 Sept. 2010
    PublisherIEEE Institute of Electrical and Electronic Engineers
    ISBN (Electronic)978-1-4244-9116-2
    ISBN (Print)978-1-4244-9117-9, 978-1-4244-9115-5
    Publication statusPublished - 2010
    MoE publication typeA4 Article in a conference publication


    • cognitive wireless mesh networks
    • energy efficiency
    • Q-learning
    • wireless communications
    • green communications


    Dive into the research topics of 'Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications'. Together they form a unique fingerprint.

    Cite this