Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective

Xianfu Chen, Celimuge Wu, Mehdi Bennis, Zhifeng Zhao, Zhu Han

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)

Abstract

In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

Original languageEnglish
Article number8674536
Pages (from-to)4262-4274
Number of pages13
JournalIEEE Transactions on Vehicular Technology
Volume68
Issue number5
DOIs
Publication statusPublished - 1 May 2019
MoE publication typeA1 Journal article-refereed

Fingerprint

Vehicular Communications
Stochastic Games
Equilibrium Solution
Online Algorithms
Game
Resources
Communication
State Space
Radio Resource Management
Unit
Network Dynamics
Online Learning
Vehicle to vehicle communications
Control Policy
Auctions
Communication Networks
Queue
Frequency allocation
Learning Algorithm
Theoretical Analysis

Keywords

  • Markov decision process
  • Markov perfect equilibrium
  • oblivious equilibrium
  • reinforcement learning
  • stochastic games
  • Vehicle-to-vehicle communications

Cite this

Chen, Xianfu ; Wu, Celimuge ; Bennis, Mehdi ; Zhao, Zhifeng ; Han, Zhu. / Learning to Entangle Radio Resources in Vehicular Communications : An Oblivious Game-Theoretic Perspective. In: IEEE Transactions on Vehicular Technology. 2019 ; Vol. 68, No. 5. pp. 4262-4274.
@article{05110cc8edce48a8b655a5bc0ee60ce1,
title = "Learning to Entangle Radio Resources in Vehicular Communications: An Oblivious Game-Theoretic Perspective",
abstract = "In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.",
keywords = "Markov decision process, Markov perfect equilibrium, oblivious equilibrium, reinforcement learning, stochastic games, Vehicle-to-vehicle communications",
author = "Xianfu Chen and Celimuge Wu and Mehdi Bennis and Zhifeng Zhao and Zhu Han",
year = "2019",
month = "5",
day = "1",
doi = "10.1109/TVT.2019.2907589",
language = "English",
volume = "68",
pages = "4262--4274",
journal = "IEEE Transactions on Vehicular Technology",
issn = "0018-9545",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
number = "5",

}

Learning to Entangle Radio Resources in Vehicular Communications : An Oblivious Game-Theoretic Perspective. / Chen, Xianfu; Wu, Celimuge; Bennis, Mehdi; Zhao, Zhifeng; Han, Zhu.

In: IEEE Transactions on Vehicular Technology, Vol. 68, No. 5, 8674536, 01.05.2019, p. 4262-4274.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Learning to Entangle Radio Resources in Vehicular Communications

T2 - An Oblivious Game-Theoretic Perspective

AU - Chen, Xianfu

AU - Wu, Celimuge

AU - Bennis, Mehdi

AU - Zhao, Zhifeng

AU - Han, Zhu

PY - 2019/5/1

Y1 - 2019/5/1

N2 - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

AB - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.

KW - Markov decision process

KW - Markov perfect equilibrium

KW - oblivious equilibrium

KW - reinforcement learning

KW - stochastic games

KW - Vehicle-to-vehicle communications

UR - http://www.scopus.com/inward/record.url?scp=85066476737&partnerID=8YFLogxK

U2 - 10.1109/TVT.2019.2907589

DO - 10.1109/TVT.2019.2907589

M3 - Article

AN - SCOPUS:85066476737

VL - 68

SP - 4262

EP - 4274

JO - IEEE Transactions on Vehicular Technology

JF - IEEE Transactions on Vehicular Technology

SN - 0018-9545

IS - 5

M1 - 8674536

ER -