Deep reinforcement learning for D2D transmission in unlicensed bands

Zhiqun Zou, Rui Yin, Xianfu Chen, Celimuge Wu

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

Original languageEnglish
Title of host publication2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019
PublisherInstitute of Electrical and Electronic Engineers IEEE
Pages42-47
ISBN (Electronic)978-1-7281-0738-7, 978-1-7281-0737-0
ISBN (Print)978-1-7281-0739-4
DOIs
Publication statusPublished - 2019
MoE publication typeA4 Article in a conference publication
Event2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019 - Changchun, China
Duration: 11 Aug 201913 Aug 2019

Conference

Conference2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019
CountryChina
CityChangchun
Period11/08/1913/08/19

Fingerprint

Wi-Fi
Convex optimization
Reinforcement learning
Reinforcement Learning
reinforcement
learning
Explosions
Q-learning
Communication
Power Allocation
Convex Optimization
Learning Process
Coexistence
Explosion
Optimization Methods
State Space
Traffic
Neural Networks
Verify
Cycle

Cite this

Zou, Z., Yin, R., Chen, X., & Wu, C. (2019). Deep reinforcement learning for D2D transmission in unlicensed bands. In 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019 (pp. 42-47). Institute of Electrical and Electronic Engineers IEEE. https://doi.org/10.1109/ICCChinaW.2019.8849971
Zou, Zhiqun ; Yin, Rui ; Chen, Xianfu ; Wu, Celimuge. / Deep reinforcement learning for D2D transmission in unlicensed bands. 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. Institute of Electrical and Electronic Engineers IEEE, 2019. pp. 42-47
@inproceedings{5db9ea7147034290b381088d83d58323,
title = "Deep reinforcement learning for D2D transmission in unlicensed bands",
abstract = "In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.",
author = "Zhiqun Zou and Rui Yin and Xianfu Chen and Celimuge Wu",
year = "2019",
doi = "10.1109/ICCChinaW.2019.8849971",
language = "English",
isbn = "978-1-7281-0739-4",
pages = "42--47",
booktitle = "2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
address = "United States",

}

Zou, Z, Yin, R, Chen, X & Wu, C 2019, Deep reinforcement learning for D2D transmission in unlicensed bands. in 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. Institute of Electrical and Electronic Engineers IEEE, pp. 42-47, 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019, Changchun, China, 11/08/19. https://doi.org/10.1109/ICCChinaW.2019.8849971

Deep reinforcement learning for D2D transmission in unlicensed bands. / Zou, Zhiqun; Yin, Rui; Chen, Xianfu; Wu, Celimuge.

2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. Institute of Electrical and Electronic Engineers IEEE, 2019. p. 42-47.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Deep reinforcement learning for D2D transmission in unlicensed bands

AU - Zou, Zhiqun

AU - Yin, Rui

AU - Chen, Xianfu

AU - Wu, Celimuge

PY - 2019

Y1 - 2019

N2 - In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

AB - In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

UR - http://www.scopus.com/inward/record.url?scp=85073605653&partnerID=8YFLogxK

U2 - 10.1109/ICCChinaW.2019.8849971

DO - 10.1109/ICCChinaW.2019.8849971

M3 - Conference article in proceedings

AN - SCOPUS:85073605653

SN - 978-1-7281-0739-4

SP - 42

EP - 47

BT - 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019

PB - Institute of Electrical and Electronic Engineers IEEE

ER -

Zou Z, Yin R, Chen X, Wu C. Deep reinforcement learning for D2D transmission in unlicensed bands. In 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. Institute of Electrical and Electronic Engineers IEEE. 2019. p. 42-47 https://doi.org/10.1109/ICCChinaW.2019.8849971