Deep reinforcement learning for D2D transmission in unlicensed bands

Zhiqun Zou, Rui Yin, Xianfu Chen, Celimuge Wu

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    Abstract

    In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

    Original languageEnglish
    Title of host publication2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019
    PublisherIEEE Institute of Electrical and Electronic Engineers
    Pages42-47
    ISBN (Electronic)978-1-7281-0738-7, 978-1-7281-0737-0
    ISBN (Print)978-1-7281-0739-4
    DOIs
    Publication statusPublished - 2019
    MoE publication typeA4 Article in a conference publication
    Event2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019 - Changchun, China
    Duration: 11 Aug 201913 Aug 2019

    Conference

    Conference2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019
    CountryChina
    CityChangchun
    Period11/08/1913/08/19

    Fingerprint

    Wi-Fi
    Convex optimization
    Reinforcement learning
    Reinforcement Learning
    reinforcement
    learning
    Explosions
    Q-learning
    Communication
    Power Allocation
    Convex Optimization
    Learning Process
    Coexistence
    Explosion
    Optimization Methods
    State Space
    Traffic
    Neural Networks
    Verify
    Cycle

    Cite this

    Zou, Z., Yin, R., Chen, X., & Wu, C. (2019). Deep reinforcement learning for D2D transmission in unlicensed bands. In 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019 (pp. 42-47). IEEE Institute of Electrical and Electronic Engineers . https://doi.org/10.1109/ICCChinaW.2019.8849971
    Zou, Zhiqun ; Yin, Rui ; Chen, Xianfu ; Wu, Celimuge. / Deep reinforcement learning for D2D transmission in unlicensed bands. 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. IEEE Institute of Electrical and Electronic Engineers , 2019. pp. 42-47
    @inproceedings{5db9ea7147034290b381088d83d58323,
    title = "Deep reinforcement learning for D2D transmission in unlicensed bands",
    abstract = "In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.",
    author = "Zhiqun Zou and Rui Yin and Xianfu Chen and Celimuge Wu",
    year = "2019",
    doi = "10.1109/ICCChinaW.2019.8849971",
    language = "English",
    isbn = "978-1-7281-0739-4",
    pages = "42--47",
    booktitle = "2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019",
    publisher = "IEEE Institute of Electrical and Electronic Engineers",
    address = "United States",

    }

    Zou, Z, Yin, R, Chen, X & Wu, C 2019, Deep reinforcement learning for D2D transmission in unlicensed bands. in 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. IEEE Institute of Electrical and Electronic Engineers , pp. 42-47, 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019, Changchun, China, 11/08/19. https://doi.org/10.1109/ICCChinaW.2019.8849971

    Deep reinforcement learning for D2D transmission in unlicensed bands. / Zou, Zhiqun; Yin, Rui; Chen, Xianfu; Wu, Celimuge.

    2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. IEEE Institute of Electrical and Electronic Engineers , 2019. p. 42-47.

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    TY - GEN

    T1 - Deep reinforcement learning for D2D transmission in unlicensed bands

    AU - Zou, Zhiqun

    AU - Yin, Rui

    AU - Chen, Xianfu

    AU - Wu, Celimuge

    PY - 2019

    Y1 - 2019

    N2 - In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

    AB - In this paper, a reinforcement learning based approach is proposed to realize the distributed power and spectrum allocation for the Device-to-Device (D2D) communications in unlicensed bands, named as D2D-U. To guarantee the harmonious coexistence with the WiFi networks, the conventional duty-cycle muting (DCM) is employed by the D2D-U links. With the proposed learning approach, D2D-U links can optimally select the time fraction on unlicensed channels without knowing the accurate WiFi traffic in a dynamic WiFi working environment. To address the state space explosion during the learning process, the Deep Q-learning network (DQN) is adopted by combining a deep neural network (DNN) with the traditional Q-learning mechanism. After obtaining the available time fraction on unlicensed channels, the spectrum and power allocation on licensed and unlicensed bands can be optimized jointly via the classic convex optimization methods at each D2D-U link. Numerical results are demonstrated to verify the effectiveness of the proposed approach.

    UR - http://www.scopus.com/inward/record.url?scp=85073605653&partnerID=8YFLogxK

    U2 - 10.1109/ICCChinaW.2019.8849971

    DO - 10.1109/ICCChinaW.2019.8849971

    M3 - Conference article in proceedings

    AN - SCOPUS:85073605653

    SN - 978-1-7281-0739-4

    SP - 42

    EP - 47

    BT - 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019

    PB - IEEE Institute of Electrical and Electronic Engineers

    ER -

    Zou Z, Yin R, Chen X, Wu C. Deep reinforcement learning for D2D transmission in unlicensed bands. In 2019 IEEE/CIC International Conference on Communications Workshops in China, ICCC Workshops 2019. IEEE Institute of Electrical and Electronic Engineers . 2019. p. 42-47 https://doi.org/10.1109/ICCChinaW.2019.8849971