TY - JOUR
T1 - Learning to Entangle Radio Resources in Vehicular Communications
T2 - An Oblivious Game-Theoretic Perspective
AU - Chen, Xianfu
AU - Wu, Celimuge
AU - Bennis, Mehdi
AU - Zhao, Zhifeng
AU - Han, Zhu
N1 - Funding Information:
This work was supported in part by the Finnish Funding Agency for Innovation (TEKES) under the Project “Wireless for Verticals (WIVE),” in part by the Academy of Finland under Grants 319759, 319758, and 289611, in part by the Telecommunications Advanced Foundation, and in part by the US MURI AFOSR MURI 18RT0073 and the National Science Foundation under Grants 1717454, 1731424, 1702850, and 1646607.
PY - 2019/5/1
Y1 - 2019/5/1
N2 - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.
AB - In this paper, we investigate non-cooperative radio resource management in a vehicle-to-vehicle communication network. The technical challenges lie in high-vehicle mobility and data traffic variations. Over the discrete scheduling slots, each vehicle user equipment (VUE)-pair competes with other VUE-pairs in the coverage of a road side unit (RSU) for the limited frequency to transmit queued data packets, aiming to optimize the expected long-term performance. The frequency allocation at the beginning of each slot at the RSU is regulated by a sealed second-price auction. Such interactions among VUE-pairs are modeled as a stochastic game with a semi-continuous global network state space. By defining a partitioned control policy, we transform the original game into an equivalent stochastic game with a global queue state space of finite size. We adopt an oblivious equilibrium (OE) to approximate the Markov perfect equilibrium, which characterizes the optimal solution to the equivalent game. The OE solution is theoretically proven to be with an asymptotic Markov equilibrium property. Due to the lack of a priori knowledge of network dynamics, we derive an online algorithm to learn the OE solution. Numerical simulations validate the theoretical analysis and show the effectiveness of the proposed online learning algorithm.
KW - Markov decision process
KW - Markov perfect equilibrium
KW - oblivious equilibrium
KW - reinforcement learning
KW - stochastic games
KW - Vehicle-to-vehicle communications
UR - http://www.scopus.com/inward/record.url?scp=85066476737&partnerID=8YFLogxK
U2 - 10.1109/TVT.2019.2907589
DO - 10.1109/TVT.2019.2907589
M3 - Article
AN - SCOPUS:85066476737
VL - 68
SP - 4262
EP - 4274
JO - IEEE Transactions on Vehicular Technology
JF - IEEE Transactions on Vehicular Technology
SN - 0018-9545
IS - 5
M1 - 8674536
ER -