TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks

R Li, Z Zhao, Xianfu Chen, J Palicot, H Zhang

Research output: Contribution to journalArticleScientificpeer-review

49 Citations (Scopus)

Abstract

Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to speed up the ongoing learning process, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jumpstart and demonstrates the feasibility of signi?cant energy ef?ciency improvement at the expense of tolerable delay performance
Original languageEnglish
Pages (from-to)2000-2011
Number of pages11
JournalIEEE Transactions on Wireless Communications
Volume13
Issue number4
DOIs
Publication statusPublished - 2014
MoE publication typeA1 Journal article-refereed

Fingerprint

Energy Saving
Base stations
Energy conservation
Traffic
Traffic Dynamics
Markov Decision Process
Reinforcement learning
Network Design
Reinforcement Learning
Expertise
Learning Process
Energy Efficiency
Energy Consumption
Energy efficiency
Forecast
Speedup
Energy utilization
Converge
Minimise
Configuration

Keywords

  • Radio access networks
  • base stations
  • sleeping mode
  • green communications
  • energy saving
  • reinforcement learning
  • transfer learning
  • actor-critic algorithm

Cite this

@article{af77c0691a2a4c469aca7d6f5ca71e5d,
title = "TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks",
abstract = "Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to speed up the ongoing learning process, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jumpstart and demonstrates the feasibility of signi?cant energy ef?ciency improvement at the expense of tolerable delay performance",
keywords = "Radio access networks, base stations, sleeping mode, green communications, energy saving, reinforcement learning, transfer learning, actor-critic algorithm",
author = "R Li and Z Zhao and Xianfu Chen and J Palicot and H Zhang",
note = "Project code: 82164",
year = "2014",
doi = "10.1109/TWC.2014.022014.130840",
language = "English",
volume = "13",
pages = "2000--2011",
journal = "IEEE Transactions on Wireless Communications",
issn = "1536-1276",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
number = "4",

}

TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks. / Li, R; Zhao, Z; Chen, Xianfu; Palicot, J; Zhang, H.

In: IEEE Transactions on Wireless Communications, Vol. 13, No. 4, 2014, p. 2000-2011.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks

AU - Li, R

AU - Zhao, Z

AU - Chen, Xianfu

AU - Palicot, J

AU - Zhang, H

N1 - Project code: 82164

PY - 2014

Y1 - 2014

N2 - Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to speed up the ongoing learning process, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jumpstart and demonstrates the feasibility of signi?cant energy ef?ciency improvement at the expense of tolerable delay performance

AB - Recent works have validated the possibility of improving energy efficiency in radio access networks (RANs), achieved by dynamically turning on/off some base stations (BSs). In this paper, we extend the research over BS switching operations, which should match up with traffic load variations. Instead of depending on the dynamic traffic loads which are still quite challenging to precisely forecast, we firstly formulate the traffic variations as a Markov decision process. Afterwards, in order to foresightedly minimize the energy consumption of RANs, we design a reinforcement learning framework based BS switching operation scheme. Furthermore, to speed up the ongoing learning process, a transfer actor-critic algorithm (TACT), which utilizes the transferred learning expertise in historical periods or neighboring regions, is proposed and provably converges. In the end, we evaluate our proposed scheme by extensive simulations under various practical configurations and show that the proposed TACT algorithm contributes to a performance jumpstart and demonstrates the feasibility of signi?cant energy ef?ciency improvement at the expense of tolerable delay performance

KW - Radio access networks

KW - base stations

KW - sleeping mode

KW - green communications

KW - energy saving

KW - reinforcement learning

KW - transfer learning

KW - actor-critic algorithm

U2 - 10.1109/TWC.2014.022014.130840

DO - 10.1109/TWC.2014.022014.130840

M3 - Article

VL - 13

SP - 2000

EP - 2011

JO - IEEE Transactions on Wireless Communications

JF - IEEE Transactions on Wireless Communications

SN - 1536-1276

IS - 4

ER -