Improving energy efficiency in green femtocell networks

A hierarchical reinforcement learning framework

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

22 Citations (Scopus)

Abstract

This paper investigates energy efficiency for two-tier femtocell networks through combining game theory and stochastic learning. With the Stackelberg game formulation, a hierarchical reinforcement learning framework is applied to studying the joint average utility maximization of macrocells and femtocells subject to the minimum signal-to-interference-plus-noise-ratio requirements. The macrocells behave as the leaders and the femtocells are followers during the learning procedure. At each time step, the leaders commit to dynamic strategies based on the best responses of the followers, while the followers compete against each other with no further information but the leaders' strategy information. In this paper, we propose two learning algorithms to schedule each cell's stochastic power levels, leading by the macrocells. Numerical experiments are presented to validate the proposed studies and show that the two learning algorithms substantially improve the energy efficiency of the femtocell networks.
Original languageEnglish
Title of host publication2013 IEEE International Conference on Communications (ICC)
PublisherInstitute of Electrical and Electronic Engineers IEEE
Pages2241-2245
ISBN (Print)978-1-4673-3122-7
DOIs
Publication statusPublished - 2013
MoE publication typeA4 Article in a conference publication
EventIEEE International Conference on Communications, ICC 2013 - Budapest, Hungary
Duration: 9 Jun 201313 Jun 2013

Conference

ConferenceIEEE International Conference on Communications, ICC 2013
Abbreviated titleICC 2013
CountryHungary
CityBudapest
Period9/06/1313/06/13

Fingerprint

Femtocell
Reinforcement learning
Energy efficiency
Learning algorithms
Game theory
Experiments

Cite this

Chen, X., Zhang, H., Chen, T., & Lasanen, M. (2013). Improving energy efficiency in green femtocell networks: A hierarchical reinforcement learning framework. In 2013 IEEE International Conference on Communications (ICC) (pp. 2241-2245). Institute of Electrical and Electronic Engineers IEEE. https://doi.org/10.1109/ICC.2013.6654861
Chen, Xianfu ; Zhang, Honggang ; Chen, Tao ; Lasanen, Mika. / Improving energy efficiency in green femtocell networks : A hierarchical reinforcement learning framework. 2013 IEEE International Conference on Communications (ICC). Institute of Electrical and Electronic Engineers IEEE, 2013. pp. 2241-2245
@inproceedings{7cc5dfbc17e14dbc9bce24f7dbd6b8e8,
title = "Improving energy efficiency in green femtocell networks: A hierarchical reinforcement learning framework",
abstract = "This paper investigates energy efficiency for two-tier femtocell networks through combining game theory and stochastic learning. With the Stackelberg game formulation, a hierarchical reinforcement learning framework is applied to studying the joint average utility maximization of macrocells and femtocells subject to the minimum signal-to-interference-plus-noise-ratio requirements. The macrocells behave as the leaders and the femtocells are followers during the learning procedure. At each time step, the leaders commit to dynamic strategies based on the best responses of the followers, while the followers compete against each other with no further information but the leaders' strategy information. In this paper, we propose two learning algorithms to schedule each cell's stochastic power levels, leading by the macrocells. Numerical experiments are presented to validate the proposed studies and show that the two learning algorithms substantially improve the energy efficiency of the femtocell networks.",
author = "Xianfu Chen and Honggang Zhang and Tao Chen and Mika Lasanen",
note = "CA2: TK810 SDA: ICT Project code: 82164",
year = "2013",
doi = "10.1109/ICC.2013.6654861",
language = "English",
isbn = "978-1-4673-3122-7",
pages = "2241--2245",
booktitle = "2013 IEEE International Conference on Communications (ICC)",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
address = "United States",

}

Chen, X, Zhang, H, Chen, T & Lasanen, M 2013, Improving energy efficiency in green femtocell networks: A hierarchical reinforcement learning framework. in 2013 IEEE International Conference on Communications (ICC). Institute of Electrical and Electronic Engineers IEEE, pp. 2241-2245, IEEE International Conference on Communications, ICC 2013, Budapest, Hungary, 9/06/13. https://doi.org/10.1109/ICC.2013.6654861

Improving energy efficiency in green femtocell networks : A hierarchical reinforcement learning framework. / Chen, Xianfu; Zhang, Honggang; Chen, Tao; Lasanen, Mika.

2013 IEEE International Conference on Communications (ICC). Institute of Electrical and Electronic Engineers IEEE, 2013. p. 2241-2245.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Improving energy efficiency in green femtocell networks

T2 - A hierarchical reinforcement learning framework

AU - Chen, Xianfu

AU - Zhang, Honggang

AU - Chen, Tao

AU - Lasanen, Mika

N1 - CA2: TK810 SDA: ICT Project code: 82164

PY - 2013

Y1 - 2013

N2 - This paper investigates energy efficiency for two-tier femtocell networks through combining game theory and stochastic learning. With the Stackelberg game formulation, a hierarchical reinforcement learning framework is applied to studying the joint average utility maximization of macrocells and femtocells subject to the minimum signal-to-interference-plus-noise-ratio requirements. The macrocells behave as the leaders and the femtocells are followers during the learning procedure. At each time step, the leaders commit to dynamic strategies based on the best responses of the followers, while the followers compete against each other with no further information but the leaders' strategy information. In this paper, we propose two learning algorithms to schedule each cell's stochastic power levels, leading by the macrocells. Numerical experiments are presented to validate the proposed studies and show that the two learning algorithms substantially improve the energy efficiency of the femtocell networks.

AB - This paper investigates energy efficiency for two-tier femtocell networks through combining game theory and stochastic learning. With the Stackelberg game formulation, a hierarchical reinforcement learning framework is applied to studying the joint average utility maximization of macrocells and femtocells subject to the minimum signal-to-interference-plus-noise-ratio requirements. The macrocells behave as the leaders and the femtocells are followers during the learning procedure. At each time step, the leaders commit to dynamic strategies based on the best responses of the followers, while the followers compete against each other with no further information but the leaders' strategy information. In this paper, we propose two learning algorithms to schedule each cell's stochastic power levels, leading by the macrocells. Numerical experiments are presented to validate the proposed studies and show that the two learning algorithms substantially improve the energy efficiency of the femtocell networks.

U2 - 10.1109/ICC.2013.6654861

DO - 10.1109/ICC.2013.6654861

M3 - Conference article in proceedings

SN - 978-1-4673-3122-7

SP - 2241

EP - 2245

BT - 2013 IEEE International Conference on Communications (ICC)

PB - Institute of Electrical and Electronic Engineers IEEE

ER -

Chen X, Zhang H, Chen T, Lasanen M. Improving energy efficiency in green femtocell networks: A hierarchical reinforcement learning framework. In 2013 IEEE International Conference on Communications (ICC). Institute of Electrical and Electronic Engineers IEEE. 2013. p. 2241-2245 https://doi.org/10.1109/ICC.2013.6654861