Multi-Beam Power Allocation in Dynamic Massive MIMO Cloud Radio Access Networks

In this paper, we investigate a cloud radio access network (Cloud-RAN) in which both fronthaul and radio access links use massive MIMO millimeter-wave (mmWave) transmissions. Such an all-mmWave Cloud-RAN architecture provides a flexible and cost-effective means for deployment of next-generation (5G and beyond) cellular networks to meet the demands of fast-growing mobile data traffic. Nevertheless, the design of transmit power allocation schemes for multiple massive MIMO beams on both the fronthaul and access links in a mmWave Cloud-RAN is challenging. In particular, the traffic and wireless channel states of multiple mobile terminals (MTs) change over time, while their statistics may not be known a priori. We formulate the joint fronthaul-access link massive MIMO beam power allocation problem as a Markov decision process (MDP) with an objective to optimize the long-term quality of service to all the MTs in a Cloud-RAN. A reinforcement learning algorithm is designed, which learns the optimal beam power allocation policy on the fly and adapts to the network dynamics. Further, by leveraging the structure of the underlying problem, a post-decision state is introduced and a function decomposition technique is developed to reduce the search space during the learning process. The evaluation results validate the convergence of our proposed scheme and demonstrate its superior performance over the state-of-the-art baselines.