Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications

Xianfu Chen, Zhifeng Zhao, Honggang Zhang, Tao Chen

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

5 Citations (Scopus)

Abstract

Cognitive wireless mesh networks have great potential to green communication. One of the critical challenges for realizing such networks is how to adaptively match transmit powers and allocate frequency resources among secondary users of the licensed frequency bands whilst maintaining the quality-of-service (QoS) requirement of the primary users, even in mutually entangled interferences environment. In this paper, we discuss the power assignment matching problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of the Signal-to-Interference-plus-Noise Ratio (SINR) requirement of each secondary user (SU), the mean-squared error (MSE) constraint by the primary users, and the energy efficiency. Due to the secondary users' selfish and spontaneous features, the problem is modeled as a stochastic non-cooperative game. We extend the conventional single-agent Q-learning to a non-cooperative multi-agent learning context, using the framework of stochastic non-cooperative games. Within the multi-agent Q-learning processes, a learning SU maintains Q-functions over joint actions set, and performs updating based on the conjecture about the other SUs' behaviors over the current Q-values. Numerical experiments on a hybrid CogMesh consisting two SUs and one specific PU suggest validity and efficiency of the proposed algorithm.
Original languageEnglish
Title of host publication2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops)
Subtitle of host publicationIstanbul, Turkey, 26-30 Sept. 2010
PublisherInstitute of Electrical and Electronic Engineers IEEE
Pages336-340
ISBN (Electronic)978-1-4244-9116-2
ISBN (Print)978-1-4244-9117-9, 978-1-4244-9115-5
DOIs
Publication statusPublished - 2010
MoE publication typeA4 Article in a conference publication

Fingerprint

Wireless mesh networks (WMN)
Communication
Frequency bands
Energy efficiency
Quality of service
Experiments

Keywords

  • cognitive wireless mesh networks
  • energy efficiency
  • Q-learning
  • wireless communications
  • green communications

Cite this

Chen, X., Zhao, Z., Zhang, H., & Chen, T. (2010). Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications. In 2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops): Istanbul, Turkey, 26-30 Sept. 2010 (pp. 336-340). Institute of Electrical and Electronic Engineers IEEE. https://doi.org/10.1109/PIMRCW.2010.5670390
Chen, Xianfu ; Zhao, Zhifeng ; Zhang, Honggang ; Chen, Tao. / Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications. 2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops): Istanbul, Turkey, 26-30 Sept. 2010. Institute of Electrical and Electronic Engineers IEEE, 2010. pp. 336-340
@inproceedings{08334279bde44509a28de5410e8633f0,
title = "Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications",
abstract = "Cognitive wireless mesh networks have great potential to green communication. One of the critical challenges for realizing such networks is how to adaptively match transmit powers and allocate frequency resources among secondary users of the licensed frequency bands whilst maintaining the quality-of-service (QoS) requirement of the primary users, even in mutually entangled interferences environment. In this paper, we discuss the power assignment matching problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of the Signal-to-Interference-plus-Noise Ratio (SINR) requirement of each secondary user (SU), the mean-squared error (MSE) constraint by the primary users, and the energy efficiency. Due to the secondary users' selfish and spontaneous features, the problem is modeled as a stochastic non-cooperative game. We extend the conventional single-agent Q-learning to a non-cooperative multi-agent learning context, using the framework of stochastic non-cooperative games. Within the multi-agent Q-learning processes, a learning SU maintains Q-functions over joint actions set, and performs updating based on the conjecture about the other SUs' behaviors over the current Q-values. Numerical experiments on a hybrid CogMesh consisting two SUs and one specific PU suggest validity and efficiency of the proposed algorithm.",
keywords = "cognitive wireless mesh networks, energy efficiency, Q-learning, wireless communications, green communications",
author = "Xianfu Chen and Zhifeng Zhao and Honggang Zhang and Tao Chen",
note = "Project code: 40668 JADEN",
year = "2010",
doi = "10.1109/PIMRCW.2010.5670390",
language = "English",
isbn = "978-1-4244-9117-9",
pages = "336--340",
booktitle = "2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops)",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
address = "United States",

}

Chen, X, Zhao, Z, Zhang, H & Chen, T 2010, Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications. in 2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops): Istanbul, Turkey, 26-30 Sept. 2010. Institute of Electrical and Electronic Engineers IEEE, pp. 336-340. https://doi.org/10.1109/PIMRCW.2010.5670390

Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications. / Chen, Xianfu; Zhao, Zhifeng; Zhang, Honggang; Chen, Tao.

2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops): Istanbul, Turkey, 26-30 Sept. 2010. Institute of Electrical and Electronic Engineers IEEE, 2010. p. 336-340.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications

AU - Chen, Xianfu

AU - Zhao, Zhifeng

AU - Zhang, Honggang

AU - Chen, Tao

N1 - Project code: 40668 JADEN

PY - 2010

Y1 - 2010

N2 - Cognitive wireless mesh networks have great potential to green communication. One of the critical challenges for realizing such networks is how to adaptively match transmit powers and allocate frequency resources among secondary users of the licensed frequency bands whilst maintaining the quality-of-service (QoS) requirement of the primary users, even in mutually entangled interferences environment. In this paper, we discuss the power assignment matching problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of the Signal-to-Interference-plus-Noise Ratio (SINR) requirement of each secondary user (SU), the mean-squared error (MSE) constraint by the primary users, and the energy efficiency. Due to the secondary users' selfish and spontaneous features, the problem is modeled as a stochastic non-cooperative game. We extend the conventional single-agent Q-learning to a non-cooperative multi-agent learning context, using the framework of stochastic non-cooperative games. Within the multi-agent Q-learning processes, a learning SU maintains Q-functions over joint actions set, and performs updating based on the conjecture about the other SUs' behaviors over the current Q-values. Numerical experiments on a hybrid CogMesh consisting two SUs and one specific PU suggest validity and efficiency of the proposed algorithm.

AB - Cognitive wireless mesh networks have great potential to green communication. One of the critical challenges for realizing such networks is how to adaptively match transmit powers and allocate frequency resources among secondary users of the licensed frequency bands whilst maintaining the quality-of-service (QoS) requirement of the primary users, even in mutually entangled interferences environment. In this paper, we discuss the power assignment matching problem in cognitive wireless mesh networks formed by a number of clusters with the consideration of the Signal-to-Interference-plus-Noise Ratio (SINR) requirement of each secondary user (SU), the mean-squared error (MSE) constraint by the primary users, and the energy efficiency. Due to the secondary users' selfish and spontaneous features, the problem is modeled as a stochastic non-cooperative game. We extend the conventional single-agent Q-learning to a non-cooperative multi-agent learning context, using the framework of stochastic non-cooperative games. Within the multi-agent Q-learning processes, a learning SU maintains Q-functions over joint actions set, and performs updating based on the conjecture about the other SUs' behaviors over the current Q-values. Numerical experiments on a hybrid CogMesh consisting two SUs and one specific PU suggest validity and efficiency of the proposed algorithm.

KW - cognitive wireless mesh networks

KW - energy efficiency

KW - Q-learning

KW - wireless communications

KW - green communications

U2 - 10.1109/PIMRCW.2010.5670390

DO - 10.1109/PIMRCW.2010.5670390

M3 - Conference article in proceedings

SN - 978-1-4244-9117-9

SN - 978-1-4244-9115-5

SP - 336

EP - 340

BT - 2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops)

PB - Institute of Electrical and Electronic Engineers IEEE

ER -

Chen X, Zhao Z, Zhang H, Chen T. Applying multi-agent Q-learning scheme in cognitive wireless mesh networks for green communications. In 2010 IEEE 21st International Symposium on Personal, Indoor and Mobile Radio Communications Workshops (PIMRC Workshops): Istanbul, Turkey, 26-30 Sept. 2010. Institute of Electrical and Electronic Engineers IEEE. 2010. p. 336-340 https://doi.org/10.1109/PIMRCW.2010.5670390