Multi-Agent Meta-Reinforcement Learning for Optimized Task Scheduling in Heterogeneous Edge Computing Systems

Liwen Niu, Xianfu Chen, Ning Zhang, Yongdong Zhu, Rui Yin, Celimuge Wu, Yangjie Cao

Research output: Contribution to journalArticleScientificpeer-review


Mobile edge computing (MEC) brings the potential to address the ever increasing computation demands from the mobile users (MUs). In addition to local processing, the resource-constrained MUs in an MEC system can also offload computation to the nearby servers for remote execution. With the explosive growth of mobile devices, computation offloading faces the challenge of spectrum congestion, which in turn deteriorates the overall quality of computation experience. This paper hence investigates computation task scheduling in a heterogeneous cellular and WiFi MEC system. Such a system provides both licensed and unlicensed spectrum opportunities. Due to the sharing of communication and computation resources as well as the uncertainties, we formulate the problem of computation task scheduling among the competing MUs in a stationary heterogeneous edge computing system as a non-cooperative stochastic game. We propose an approximation-based multi-agent Markov decision process without the global system state observations, under which a multi-agent proximal policy optimization (PPO) algorithm is derived to solve the corresponding Nash equilibrium. When expanding to a non-stationary heterogeneous edge computing system, the obtained algorithm suffers from the slow convergence due to constrained adaptability. Accordingly, we explore meta-learning and propose a multi-agent meta-PPO algorithm, which rapidly adapts the control policy learning to the non-stationarity. Numerical experiments demonstrate performance gains from our proposed algorithms.

Original languageEnglish
Pages (from-to)1
Number of pages14
JournalIEEE Internet of Things Journal
Publication statusE-pub ahead of print - 31 Jan 2023
MoE publication typeA1 Journal article-refereed


  • computation task scheduling
  • Edge computing
  • Heterogeneous edge computing systems
  • Markov decision process
  • meta-learning
  • multi-agent proximal policy optimization
  • Processor scheduling
  • Scheduling
  • Servers
  • Task analysis
  • Training
  • Wireless fidelity


Dive into the research topics of 'Multi-Agent Meta-Reinforcement Learning for Optimized Task Scheduling in Heterogeneous Edge Computing Systems'. Together they form a unique fingerprint.

Cite this