Using machine learning in the adaptive control of a smart environment

Master's thesis

Sakari Stenudd

Research output: Book/ReportBook (author)Scientificpeer-review

Abstract

The purpose of this thesis is to study the possibilities and need for utilising machine learning in a smart environment. The most important goal of smart environments is to improve the experience of their inhabitants. This requires adaptation to the behaviour of the users and the other changing conditions in the environment. Hence, the achievement of functional adaptation requires finding a way to change the behaviour of the environment according to the changed user behaviour and other conditions. Machine learning is a research area that studies the techniques which make it possible for software agents to improve their operation over time. The research method chosen in this thesis was to review existing smart environment projects and to analyse the usages of machine learning within them. Based upon these uses, a model for using machine learning in a smart environment was created. As a result, four different categories of machine learning in smart environments were identified: prediction, recognition, detection and optimisation. When deployed to the environment, these categories form a clear loop structure in which the outputs of previous learning agents serve as inputs for the next agents, which ultimately enables the making of changes to the environment according to its current state. This kind of loop is called a control loop in adaptive systems. To evaluate the suitability of the model for using machine learning in a smart environment, two demonstrations were carried out in an environment using a Smart-M3 inter-operability platform, both utilising machine learning in one of the above-discussed categories. In the first experiment neural networks were used to predict query latencies in different situations in the environment. The predictions of the network were compared to the outputs of two simpler models. The results showed that the neural network approach was capable of adapting to rapid changes more quickly. However, it also made more false assumptions about the impact of the different parameters. The second experiment belongs to the optimisation category. In this experiment a decision maker was implemented for a resource allocation problem in a distributed multi-media streaming application. It used reinforcement learning with a look-up table and an implementation of the Q-learning algorithm. After the learning period the agent was capable of making optimal decisions. The experiments confirm that it is suitable to use the model described in this thesis in smart environments. The model includes the most important uses of machine learning and it is consistent with other results in the areas of smart environments and self-adaptive software.
Original languageEnglish
Place of PublicationEspoo
PublisherVTT Technical Research Centre of Finland
Number of pages82
ISBN (Electronic)978-951-38-7420-9
Publication statusPublished - 2010
MoE publication typeC1 Separate scientific books

Publication series

NameVTT Publications
PublisherVTT
No.751
ISSN (Print)0357-9387
ISSN (Electronic)1455-0849

Fingerprint

Learning systems
Experiments
Neural networks
Media streaming
Software agents
Adaptive systems
Reinforcement learning
Interoperability
Learning algorithms
Resource allocation
Demonstrations

Keywords

  • smart space
  • inter-operability
  • control loop
  • adaptive systems
  • self-adaptive software
  • reinforcement learning
  • Smart-M3 IOP

Cite this

Stenudd, S. (2010). Using machine learning in the adaptive control of a smart environment: Master's thesis. Espoo: VTT Technical Research Centre of Finland. VTT Publications, No. 751
Stenudd, Sakari. / Using machine learning in the adaptive control of a smart environment : Master's thesis. Espoo : VTT Technical Research Centre of Finland, 2010. 82 p. (VTT Publications; No. 751).
@book{1dfc7d6c2769458ba8c41908ffad4615,
title = "Using machine learning in the adaptive control of a smart environment: Master's thesis",
abstract = "The purpose of this thesis is to study the possibilities and need for utilising machine learning in a smart environment. The most important goal of smart environments is to improve the experience of their inhabitants. This requires adaptation to the behaviour of the users and the other changing conditions in the environment. Hence, the achievement of functional adaptation requires finding a way to change the behaviour of the environment according to the changed user behaviour and other conditions. Machine learning is a research area that studies the techniques which make it possible for software agents to improve their operation over time. The research method chosen in this thesis was to review existing smart environment projects and to analyse the usages of machine learning within them. Based upon these uses, a model for using machine learning in a smart environment was created. As a result, four different categories of machine learning in smart environments were identified: prediction, recognition, detection and optimisation. When deployed to the environment, these categories form a clear loop structure in which the outputs of previous learning agents serve as inputs for the next agents, which ultimately enables the making of changes to the environment according to its current state. This kind of loop is called a control loop in adaptive systems. To evaluate the suitability of the model for using machine learning in a smart environment, two demonstrations were carried out in an environment using a Smart-M3 inter-operability platform, both utilising machine learning in one of the above-discussed categories. In the first experiment neural networks were used to predict query latencies in different situations in the environment. The predictions of the network were compared to the outputs of two simpler models. The results showed that the neural network approach was capable of adapting to rapid changes more quickly. However, it also made more false assumptions about the impact of the different parameters. The second experiment belongs to the optimisation category. In this experiment a decision maker was implemented for a resource allocation problem in a distributed multi-media streaming application. It used reinforcement learning with a look-up table and an implementation of the Q-learning algorithm. After the learning period the agent was capable of making optimal decisions. The experiments confirm that it is suitable to use the model described in this thesis in smart environments. The model includes the most important uses of machine learning and it is consistent with other results in the areas of smart environments and self-adaptive software.",
keywords = "smart space, inter-operability, control loop, adaptive systems, self-adaptive software, reinforcement learning, Smart-M3 IOP",
author = "Sakari Stenudd",
year = "2010",
language = "English",
series = "VTT Publications",
publisher = "VTT Technical Research Centre of Finland",
number = "751",
address = "Finland",

}

Stenudd, S 2010, Using machine learning in the adaptive control of a smart environment: Master's thesis. VTT Publications, no. 751, VTT Technical Research Centre of Finland, Espoo.

Using machine learning in the adaptive control of a smart environment : Master's thesis. / Stenudd, Sakari.

Espoo : VTT Technical Research Centre of Finland, 2010. 82 p. (VTT Publications; No. 751).

Research output: Book/ReportBook (author)Scientificpeer-review

TY - BOOK

T1 - Using machine learning in the adaptive control of a smart environment

T2 - Master's thesis

AU - Stenudd, Sakari

PY - 2010

Y1 - 2010

N2 - The purpose of this thesis is to study the possibilities and need for utilising machine learning in a smart environment. The most important goal of smart environments is to improve the experience of their inhabitants. This requires adaptation to the behaviour of the users and the other changing conditions in the environment. Hence, the achievement of functional adaptation requires finding a way to change the behaviour of the environment according to the changed user behaviour and other conditions. Machine learning is a research area that studies the techniques which make it possible for software agents to improve their operation over time. The research method chosen in this thesis was to review existing smart environment projects and to analyse the usages of machine learning within them. Based upon these uses, a model for using machine learning in a smart environment was created. As a result, four different categories of machine learning in smart environments were identified: prediction, recognition, detection and optimisation. When deployed to the environment, these categories form a clear loop structure in which the outputs of previous learning agents serve as inputs for the next agents, which ultimately enables the making of changes to the environment according to its current state. This kind of loop is called a control loop in adaptive systems. To evaluate the suitability of the model for using machine learning in a smart environment, two demonstrations were carried out in an environment using a Smart-M3 inter-operability platform, both utilising machine learning in one of the above-discussed categories. In the first experiment neural networks were used to predict query latencies in different situations in the environment. The predictions of the network were compared to the outputs of two simpler models. The results showed that the neural network approach was capable of adapting to rapid changes more quickly. However, it also made more false assumptions about the impact of the different parameters. The second experiment belongs to the optimisation category. In this experiment a decision maker was implemented for a resource allocation problem in a distributed multi-media streaming application. It used reinforcement learning with a look-up table and an implementation of the Q-learning algorithm. After the learning period the agent was capable of making optimal decisions. The experiments confirm that it is suitable to use the model described in this thesis in smart environments. The model includes the most important uses of machine learning and it is consistent with other results in the areas of smart environments and self-adaptive software.

AB - The purpose of this thesis is to study the possibilities and need for utilising machine learning in a smart environment. The most important goal of smart environments is to improve the experience of their inhabitants. This requires adaptation to the behaviour of the users and the other changing conditions in the environment. Hence, the achievement of functional adaptation requires finding a way to change the behaviour of the environment according to the changed user behaviour and other conditions. Machine learning is a research area that studies the techniques which make it possible for software agents to improve their operation over time. The research method chosen in this thesis was to review existing smart environment projects and to analyse the usages of machine learning within them. Based upon these uses, a model for using machine learning in a smart environment was created. As a result, four different categories of machine learning in smart environments were identified: prediction, recognition, detection and optimisation. When deployed to the environment, these categories form a clear loop structure in which the outputs of previous learning agents serve as inputs for the next agents, which ultimately enables the making of changes to the environment according to its current state. This kind of loop is called a control loop in adaptive systems. To evaluate the suitability of the model for using machine learning in a smart environment, two demonstrations were carried out in an environment using a Smart-M3 inter-operability platform, both utilising machine learning in one of the above-discussed categories. In the first experiment neural networks were used to predict query latencies in different situations in the environment. The predictions of the network were compared to the outputs of two simpler models. The results showed that the neural network approach was capable of adapting to rapid changes more quickly. However, it also made more false assumptions about the impact of the different parameters. The second experiment belongs to the optimisation category. In this experiment a decision maker was implemented for a resource allocation problem in a distributed multi-media streaming application. It used reinforcement learning with a look-up table and an implementation of the Q-learning algorithm. After the learning period the agent was capable of making optimal decisions. The experiments confirm that it is suitable to use the model described in this thesis in smart environments. The model includes the most important uses of machine learning and it is consistent with other results in the areas of smart environments and self-adaptive software.

KW - smart space

KW - inter-operability

KW - control loop

KW - adaptive systems

KW - self-adaptive software

KW - reinforcement learning

KW - Smart-M3 IOP

M3 - Book (author)

T3 - VTT Publications

BT - Using machine learning in the adaptive control of a smart environment

PB - VTT Technical Research Centre of Finland

CY - Espoo

ER -

Stenudd S. Using machine learning in the adaptive control of a smart environment: Master's thesis. Espoo: VTT Technical Research Centre of Finland, 2010. 82 p. (VTT Publications; No. 751).