Learning to Schedule Resources in Software-Defined Radio Access Networks

A software-defined control plane simplifies network operations in dense radio access networks (RANs) by abstracting the base stations as a logical centralized network controller (CNC). In a software-defined RAN, the CNC and the wireless service providers (WSPs) can thus be decoupled. The CNC allocates subbands to the mobile terminals (MTs) based on their submitted bids. Such an auction is repeated across time and regulated by the Vickrey-Clarke-Groves pricing mechanism. The objective of an MT subscribed to a particular WSP is to optimize the expected long-term transmit power in transmitting packets subject to a specific Quality-of-Service constraint. We formulate the problem as a multi-agent Markov decision process, where the subband allocation (SA) and packet scheduling decisions are a function of the global network state. To address the challenges of signalling overhead and computational complexity, we approximate the queue state-SA factor by the sum of per-MT queue state value functions, and derive an online localized algorithm to learn them. The presented experiments show significant performance gains from our proposed studies.