Conjectural variations in multi-agent reinforcement learning for energy-efficient cognitive wireless mesh networks

As energy saving and environmental protection become an inevitable trend, researchers need to shift their focus to "green" oriented architecture design. Recent advances in the area of cognitive radio (CR) have significant potential towards "green" communications. One of the critical challenges for operating CRs in a wireless mesh network is how to efficiently allocate transmission powers and frequency resource among the secondary users (SUs) while satisfying the quality-of-service constraints of primary users. Due to the SUs' intelligent and selfish properties, this paper focuses on the non-cooperative spectrum sharing in cognitive wireless mesh networks formed by a number of clusters. In order to study the competition behaviors of SUs in a dynamic environment, the problem is modeled as a stochastic learning process. We first extend the single-agent reinforcement learning (RL) to a multi-user context, based on which a conjecture based multi-agent RL algorithm is proposed. A rational SU learns the optimal transmission strategy from the conjecture over the other SUs' responses