Information Freshness Optimization in UAV-aided Vehicular Metaverse: A PPO-based Learning Approach

The digital twin technology facilitates the application of Metaverse in autonomous driving. Particularly, this paper focuses on investigating an unmanned aerial vehicle (UAV)-aided vehicular Metaverse. In specific, the moving vehicles in physical world collect the real-time traffic data, which is synchronized through the UAV to the virtual world to help the autonomous vehicle (AV) simulation. For such a physical-virtual synchro-nization process, we define the age of incorrect information (AOII) to measure the traffic data freshness. Accounting for the randomness in the physical world, we jointly optimize the UAV trajectory, the vehicle scheduling and the semantic extraction of collected data under the Markov decision process (MDP) framework. Our objective is to minimize the expected long-term system AOII. Without the statistical knowledge of physical-world randomness, we propose to leverage a proximal policy optimization based deep reinforcement learning algorithm to solve the optimal control policy to the MDP formulation. We conduct numerical experiments to verify the accuracy of the theoretical analysis, and the results demonstrate the performance gains from our proposed algorithm.