This paper focuses on a vehicle-to-vehicle communication network, for which we study the radio resource management problem accounting for the dynamics of both vehicle mobility and traffic variations. Across the discrete scheduling slots, the vehicle user equipment (VUE)-pairs covered by a roadside unit (RSU) compete with each other for the limited frequency resource, aiming to optimize the long-term performance. We model such a competitive decision-making process as a stochastic game. Due to the semi-continuous state space, we define a partitioned control policy, which facilitates the transformation of the stochastic game into an equivalent stochastic game. For the equivalent stochastic game with a large number of VUE-pairs, the Markov perfect equilibrium can be well approximated using an oblivious equilibrium (OE). Moreover, to avoid the dependence on the statistical information of network dynamics, an online reinforcement learning scheme is obtained to arrive at the OE solution. Simulations validate the theoretical studies performed in this paper.