Abstract
In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.
| Original language | English |
|---|---|
| Article number | 8674536 |
| Pages (from-to) | 4262-4274 |
| Number of pages | 13 |
| Journal | IEEE Transactions on Vehicular Technology |
| Volume | 68 |
| Issue number | 5 |
| DOIs | |
| Publication status | Published - 1 May 2019 |
| MoE publication type | A1 Journal article-refereed |
Fingerprint
Keywords
- Markov decision process
- Markov perfect equilibrium
- oblivious equilibrium
- reinforcement learning
- stochastic games
- Vehicle-to-vehicle communications
Cite this
}
Learning to Entangle Radio Resources in Vehicular Communications : An Oblivious Game-Theoretic Perspective. / Chen, Xianfu; Wu, Celimuge; Bennis, Mehdi; Zhao, Zhifeng; Han, Zhu.
In: IEEE Transactions on Vehicular Technology, Vol. 68, No. 5, 8674536, 01.05.2019, p. 4262-4274.Research output: Contribution to journal › Article › Scientific › peer-review
TY - JOUR
T1 - Learning to Entangle Radio Resources in Vehicular Communications
T2 - An Oblivious Game-Theoretic Perspective
AU - Chen, Xianfu
AU - Wu, Celimuge
AU - Bennis, Mehdi
AU - Zhao, Zhifeng
AU - Han, Zhu
PY - 2019/5/1
Y1 - 2019/5/1
N2 - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.
AB - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.
KW - Markov decision process
KW - Markov perfect equilibrium
KW - oblivious equilibrium
KW - reinforcement learning
KW - stochastic games
KW - Vehicle-to-vehicle communications
UR - http://www.scopus.com/inward/record.url?scp=85066476737&partnerID=8YFLogxK
U2 - 10.1109/TVT.2019.2907589
DO - 10.1109/TVT.2019.2907589
M3 - Article
VL - 68
SP - 4262
EP - 4274
JO - IEEE Transactions on Vehicular Technology
JF - IEEE Transactions on Vehicular Technology
SN - 0018-9545
IS - 5
M1 - 8674536
ER -