Reinforcement learning for control and optimization of real buildings: Identifying and addressing implementation hurdles

As part of the broader digitalization of society, artificial intelligence is one promising way to improve and enhance energy use in buildings. Traditionally, the building service systems have been managed with manually tuned rule-based controllers. While they are cost-efficient and simple, they might suffer from suboptimality and lack of adaptivity to the changes in the surrounding environment. Reinforcement learning (RL) is a machine learning type well-suited for control and optimization tasks. In RL, optimal control is approached through trial and error, with the goal of maximizing cumulative rewards. In simulation studies, it has shown great potential for the control and optimization of building applications. While a lot of research has been conducted in this area, real-world implementations are harder to find. After concluding that AI-based market solutions exist, this review summarizes how thoroughly these applications have been tested on practical deployments and how these studies have addressed securing safety of the adaptive solutions and possibility to utilize the solutions on a wider scale. Digital twin solutions could enable pretraining and online deployment. Additionally, the paper evaluates how the ethical aspects are addressed in these implementations and concludes that the RL-control systems lack a systematic ethical analysis. Lack of long-term studies and the focus only on just heating or cooling instead of more holistic view considering the entire building management system leaves significant gaps in understanding the reliability and effectiveness of the proposed RL solutions in complex real-world settings.