Reinforcement learning enhanced iterative power allocation in stochastic cognitive wireless mesh networks

Xianfu Chen (Corresponding Author), Zhifeng Zhao, Honggang Zhang, Tao Chen

Research output: Contribution to journalArticleScientificpeer-review

10 Citations (Scopus)

Abstract

As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the licensed frequency bands. One of the critical challenges for realizing such network is how to adaptively allocate transmit powers and frequency resources among secondary users (SUs) of the licensed frequency bands while maintaining the quality-of-service (QoS) requirement of the primary users (PUs). In this paper, we consider the power control problem in the context of cognitive wireless mesh networks formed by a number of clusters under the total transmit power constraint by each SU as well as the mean-squared error (MSE) constraint by PUs. The problem is modeled as a non-cooperative game. A distributed iterative power allocation algorithm is designed to reach the Nash equilibrium (NE) between the coexisting interfered links. It offers an opportunity for SUs to negotiate the best use of power and frequency with each other. Furthermore, how to adaptively negotiate the transmission power level and spectrum usage among the SUs according to the changing networking environment is discussed. We present an intelligent policy based on reinforcement learning to acquire the stochastic behavior of PUs. Based on the learning approach, the SUs can adapt to the dynamics of the interference environment state and reach new NEs quickly through partially cooperative information sharing via a common control channel. Theoretical analysis and numerical results both show effectiveness of the intelligent policy.
Original languageEnglish
Pages (from-to)89-104
Number of pages16
JournalWireless Personal Communications
Volume57
Issue number1
DOIs
Publication statusPublished - 2010
MoE publication typeA1 Journal article-refereed

Fingerprint

Wireless mesh networks (WMN)
Reinforcement learning
Frequency bands
Power transmission
Power control
Quality of service

Keywords

  • CogMesh
  • Cognitive radio
  • Cognitive wireless mesh networks
  • Distributed power control
  • Dynamic programming
  • Dynamic spectrum access
  • Iterative water-filling
  • Reinforcement learning

Cite this

@article{bc39490430524281b7caaade51a2c9c0,
title = "Reinforcement learning enhanced iterative power allocation in stochastic cognitive wireless mesh networks",
abstract = "As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the licensed frequency bands. One of the critical challenges for realizing such network is how to adaptively allocate transmit powers and frequency resources among secondary users (SUs) of the licensed frequency bands while maintaining the quality-of-service (QoS) requirement of the primary users (PUs). In this paper, we consider the power control problem in the context of cognitive wireless mesh networks formed by a number of clusters under the total transmit power constraint by each SU as well as the mean-squared error (MSE) constraint by PUs. The problem is modeled as a non-cooperative game. A distributed iterative power allocation algorithm is designed to reach the Nash equilibrium (NE) between the coexisting interfered links. It offers an opportunity for SUs to negotiate the best use of power and frequency with each other. Furthermore, how to adaptively negotiate the transmission power level and spectrum usage among the SUs according to the changing networking environment is discussed. We present an intelligent policy based on reinforcement learning to acquire the stochastic behavior of PUs. Based on the learning approach, the SUs can adapt to the dynamics of the interference environment state and reach new NEs quickly through partially cooperative information sharing via a common control channel. Theoretical analysis and numerical results both show effectiveness of the intelligent policy.",
keywords = "CogMesh, Cognitive radio, Cognitive wireless mesh networks, Distributed power control, Dynamic programming, Dynamic spectrum access, Iterative water-filling, Reinforcement learning",
author = "Xianfu Chen and Zhifeng Zhao and Honggang Zhang and Tao Chen",
year = "2010",
doi = "10.1007/s11277-010-0008-6",
language = "English",
volume = "57",
pages = "89--104",
journal = "Wireless Personal Communications",
issn = "0929-6212",
publisher = "Springer",
number = "1",

}

Reinforcement learning enhanced iterative power allocation in stochastic cognitive wireless mesh networks. / Chen, Xianfu (Corresponding Author); Zhao, Zhifeng; Zhang, Honggang; Chen, Tao.

In: Wireless Personal Communications, Vol. 57, No. 1, 2010, p. 89-104.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Reinforcement learning enhanced iterative power allocation in stochastic cognitive wireless mesh networks

AU - Chen, Xianfu

AU - Zhao, Zhifeng

AU - Zhang, Honggang

AU - Chen, Tao

PY - 2010

Y1 - 2010

N2 - As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the licensed frequency bands. One of the critical challenges for realizing such network is how to adaptively allocate transmit powers and frequency resources among secondary users (SUs) of the licensed frequency bands while maintaining the quality-of-service (QoS) requirement of the primary users (PUs). In this paper, we consider the power control problem in the context of cognitive wireless mesh networks formed by a number of clusters under the total transmit power constraint by each SU as well as the mean-squared error (MSE) constraint by PUs. The problem is modeled as a non-cooperative game. A distributed iterative power allocation algorithm is designed to reach the Nash equilibrium (NE) between the coexisting interfered links. It offers an opportunity for SUs to negotiate the best use of power and frequency with each other. Furthermore, how to adaptively negotiate the transmission power level and spectrum usage among the SUs according to the changing networking environment is discussed. We present an intelligent policy based on reinforcement learning to acquire the stochastic behavior of PUs. Based on the learning approach, the SUs can adapt to the dynamics of the interference environment state and reach new NEs quickly through partially cooperative information sharing via a common control channel. Theoretical analysis and numerical results both show effectiveness of the intelligent policy.

AB - As the scarce spectrum resource is becoming over-crowded, cognitive wireless mesh networks have great flexibility to improve the spectrum utilization by opportunistically accessing the licensed frequency bands. One of the critical challenges for realizing such network is how to adaptively allocate transmit powers and frequency resources among secondary users (SUs) of the licensed frequency bands while maintaining the quality-of-service (QoS) requirement of the primary users (PUs). In this paper, we consider the power control problem in the context of cognitive wireless mesh networks formed by a number of clusters under the total transmit power constraint by each SU as well as the mean-squared error (MSE) constraint by PUs. The problem is modeled as a non-cooperative game. A distributed iterative power allocation algorithm is designed to reach the Nash equilibrium (NE) between the coexisting interfered links. It offers an opportunity for SUs to negotiate the best use of power and frequency with each other. Furthermore, how to adaptively negotiate the transmission power level and spectrum usage among the SUs according to the changing networking environment is discussed. We present an intelligent policy based on reinforcement learning to acquire the stochastic behavior of PUs. Based on the learning approach, the SUs can adapt to the dynamics of the interference environment state and reach new NEs quickly through partially cooperative information sharing via a common control channel. Theoretical analysis and numerical results both show effectiveness of the intelligent policy.

KW - CogMesh

KW - Cognitive radio

KW - Cognitive wireless mesh networks

KW - Distributed power control

KW - Dynamic programming

KW - Dynamic spectrum access

KW - Iterative water-filling

KW - Reinforcement learning

U2 - 10.1007/s11277-010-0008-6

DO - 10.1007/s11277-010-0008-6

M3 - Article

VL - 57

SP - 89

EP - 104

JO - Wireless Personal Communications

JF - Wireless Personal Communications

SN - 0929-6212

IS - 1

ER -