Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective

Xianfu Chen, Celimuge Wu, Mehdi Bennis, Zhifeng Zhao, Zhu Han

    Research output: Contribution to journalArticleScientificpeer-review

    3 Citations (Scopus)

    Abstract

    In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

    Original languageEnglish
    Article number8674536
    Pages (from-to)4262-4274
    Number of pages13
    JournalIEEE Transactions on Vehicular Technology
    Volume68
    Issue number5
    DOIs
    Publication statusPublished - 1 May 2019
    MoE publication typeA1 Journal article-refereed

    Fingerprint

    Vehicular Communications
    Stochastic Games
    Equilibrium Solution
    Online Algorithms
    Game
    Resources
    Communication
    State Space
    Radio Resource Management
    Unit
    Network Dynamics
    Online Learning
    Vehicle to vehicle communications
    Control Policy
    Auctions
    Communication Networks
    Queue
    Frequency allocation
    Learning Algorithm
    Theoretical Analysis

    Keywords

    • Markov decision process
    • Markov perfect equilibrium
    • oblivious equilibrium
    • reinforcement learning
    • stochastic games
    • Vehicle-to-vehicle communications

    Cite this

    Chen, Xianfu ; Wu, Celimuge ; Bennis, Mehdi ; Zhao, Zhifeng ; Han, Zhu. / Learning to Entangle Radio Resources in Vehicular Communications : An Oblivious Game-Theoretic Perspective. In: IEEE Transactions on Vehicular Technology. 2019 ; Vol. 68, No. 5. pp. 4262-4274.
    @article{05110cc8edce48a8b655a5bc0ee60ce1,
    title = "Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective",
    abstract = "In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.",
    keywords = "Markov decision process, Markov perfect equilibrium, oblivious equilibrium, reinforcement learning, stochastic games, Vehicle-to-vehicle communications",
    author = "Xianfu Chen and Celimuge Wu and Mehdi Bennis and Zhifeng Zhao and Zhu Han",
    year = "2019",
    month = "5",
    day = "1",
    doi = "10.1109/TVT.2019.2907589",
    language = "English",
    volume = "68",
    pages = "4262--4274",
    journal = "IEEE Transactions on Vehicular Technology",
    issn = "0018-9545",
    publisher = "IEEE Institute of Electrical and Electronic Engineers",
    number = "5",

    }

    Learning to Entangle Radio Resources in Vehicular Communications : An Oblivious Game-Theoretic Perspective. / Chen, Xianfu; Wu, Celimuge; Bennis, Mehdi; Zhao, Zhifeng; Han, Zhu.

    In: IEEE Transactions on Vehicular Technology, Vol. 68, No. 5, 8674536, 01.05.2019, p. 4262-4274.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - Learning to Entangle Radio Resources in Vehicular Communications

    T2 - An Oblivious Game-Theoretic Perspective

    AU - Chen, Xianfu

    AU - Wu, Celimuge

    AU - Bennis, Mehdi

    AU - Zhao, Zhifeng

    AU - Han, Zhu

    PY - 2019/5/1

    Y1 - 2019/5/1

    N2 - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

    AB - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

    KW - Markov decision process

    KW - Markov perfect equilibrium

    KW - oblivious equilibrium

    KW - reinforcement learning

    KW - stochastic games

    KW - Vehicle-to-vehicle communications

    UR - http://www.scopus.com/inward/record.url?scp=85066476737&partnerID=8YFLogxK

    U2 - 10.1109/TVT.2019.2907589

    DO - 10.1109/TVT.2019.2907589

    M3 - Article

    AN - SCOPUS:85066476737

    VL - 68

    SP - 4262

    EP - 4274

    JO - IEEE Transactions on Vehicular Technology

    JF - IEEE Transactions on Vehicular Technology

    SN - 0018-9545

    IS - 5

    M1 - 8674536

    ER -