Cognitive wireless access selection at client side: Performance study of a Q-learning approach

Olli Mämmelä, Petteri Mannersalo

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

2 Citations (Scopus)

Abstract

The high dynamics of mobile and wireless networks calls for intelligent mechanisms to select access networks and corresponding points of access for the clients and their active applications. However, one needs to be careful not to increase the number of handovers substantially as it may cause large communication overhead to the network. In this paper, we consider mechanisms located at the client-side where the greedy selfish behavior should be regulated by using algorithms which simultaneously improve the quality of experience (QoE) but do not disturb much or, in the best case, even improve the overall network performance. Specifically, we introduce a Q-learning based QoE-aware access selection algorithm which enables the clients to learn from past experiences in order to find the optimal actions. The statuses of the available points of access are described by a cascade fuzzy classifier. The Q-learning based solution is compared to the default mechanism and an opportunistic fuzzy inference algorithm by simulation. The results indicate that a Q-learning approach is able to keep the number of handovers reasonably low while still achieving a good QoE, thus providing a better approach both from the user and the network operator perspective
Original languageEnglish
Title of host publicationProceedings
Subtitle of host publicationIEEE Network Operations and Management Symposium, NOMS 2014
PublisherInstitute of Electrical and Electronic Engineers IEEE
Number of pages4
ISBN (Print)978-147990913-1
DOIs
Publication statusPublished - 2014
MoE publication typeA4 Article in a conference publication
Event14th IEEE/IFIP Network Operations and Management Symposium: Management in a Software Defined World, NOMS 2014 - Krakow, Poland
Duration: 5 May 20149 May 2014

Conference

Conference14th IEEE/IFIP Network Operations and Management Symposium: Management in a Software Defined World, NOMS 2014
Abbreviated titleNOMS 2014
CountryPoland
CityKrakow
Period5/05/149/05/14

Fingerprint

Wireless networks
Cascades (fluid mechanics)
Fuzzy inference
Network performance
Classifiers
Communication

Cite this

Mämmelä, O., & Mannersalo, P. (2014). Cognitive wireless access selection at client side: Performance study of a Q-learning approach. In Proceedings: IEEE Network Operations and Management Symposium, NOMS 2014 [6838343] Institute of Electrical and Electronic Engineers IEEE. https://doi.org/10.1109/NOMS.2014.6838343
Mämmelä, Olli ; Mannersalo, Petteri. / Cognitive wireless access selection at client side : Performance study of a Q-learning approach. Proceedings: IEEE Network Operations and Management Symposium, NOMS 2014. Institute of Electrical and Electronic Engineers IEEE, 2014.
@inproceedings{fbc5f41ddaad48219afe14d009edc5c7,
title = "Cognitive wireless access selection at client side: Performance study of a Q-learning approach",
abstract = "The high dynamics of mobile and wireless networks calls for intelligent mechanisms to select access networks and corresponding points of access for the clients and their active applications. However, one needs to be careful not to increase the number of handovers substantially as it may cause large communication overhead to the network. In this paper, we consider mechanisms located at the client-side where the greedy selfish behavior should be regulated by using algorithms which simultaneously improve the quality of experience (QoE) but do not disturb much or, in the best case, even improve the overall network performance. Specifically, we introduce a Q-learning based QoE-aware access selection algorithm which enables the clients to learn from past experiences in order to find the optimal actions. The statuses of the available points of access are described by a cascade fuzzy classifier. The Q-learning based solution is compared to the default mechanism and an opportunistic fuzzy inference algorithm by simulation. The results indicate that a Q-learning approach is able to keep the number of handovers reasonably low while still achieving a good QoE, thus providing a better approach both from the user and the network operator perspective",
author = "Olli M{\"a}mmel{\"a} and Petteri Mannersalo",
year = "2014",
doi = "10.1109/NOMS.2014.6838343",
language = "English",
isbn = "978-147990913-1",
booktitle = "Proceedings",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
address = "United States",

}

Mämmelä, O & Mannersalo, P 2014, Cognitive wireless access selection at client side: Performance study of a Q-learning approach. in Proceedings: IEEE Network Operations and Management Symposium, NOMS 2014., 6838343, Institute of Electrical and Electronic Engineers IEEE, 14th IEEE/IFIP Network Operations and Management Symposium: Management in a Software Defined World, NOMS 2014, Krakow, Poland, 5/05/14. https://doi.org/10.1109/NOMS.2014.6838343

Cognitive wireless access selection at client side : Performance study of a Q-learning approach. / Mämmelä, Olli; Mannersalo, Petteri.

Proceedings: IEEE Network Operations and Management Symposium, NOMS 2014. Institute of Electrical and Electronic Engineers IEEE, 2014. 6838343.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Cognitive wireless access selection at client side

T2 - Performance study of a Q-learning approach

AU - Mämmelä, Olli

AU - Mannersalo, Petteri

PY - 2014

Y1 - 2014

N2 - The high dynamics of mobile and wireless networks calls for intelligent mechanisms to select access networks and corresponding points of access for the clients and their active applications. However, one needs to be careful not to increase the number of handovers substantially as it may cause large communication overhead to the network. In this paper, we consider mechanisms located at the client-side where the greedy selfish behavior should be regulated by using algorithms which simultaneously improve the quality of experience (QoE) but do not disturb much or, in the best case, even improve the overall network performance. Specifically, we introduce a Q-learning based QoE-aware access selection algorithm which enables the clients to learn from past experiences in order to find the optimal actions. The statuses of the available points of access are described by a cascade fuzzy classifier. The Q-learning based solution is compared to the default mechanism and an opportunistic fuzzy inference algorithm by simulation. The results indicate that a Q-learning approach is able to keep the number of handovers reasonably low while still achieving a good QoE, thus providing a better approach both from the user and the network operator perspective

AB - The high dynamics of mobile and wireless networks calls for intelligent mechanisms to select access networks and corresponding points of access for the clients and their active applications. However, one needs to be careful not to increase the number of handovers substantially as it may cause large communication overhead to the network. In this paper, we consider mechanisms located at the client-side where the greedy selfish behavior should be regulated by using algorithms which simultaneously improve the quality of experience (QoE) but do not disturb much or, in the best case, even improve the overall network performance. Specifically, we introduce a Q-learning based QoE-aware access selection algorithm which enables the clients to learn from past experiences in order to find the optimal actions. The statuses of the available points of access are described by a cascade fuzzy classifier. The Q-learning based solution is compared to the default mechanism and an opportunistic fuzzy inference algorithm by simulation. The results indicate that a Q-learning approach is able to keep the number of handovers reasonably low while still achieving a good QoE, thus providing a better approach both from the user and the network operator perspective

U2 - 10.1109/NOMS.2014.6838343

DO - 10.1109/NOMS.2014.6838343

M3 - Conference article in proceedings

SN - 978-147990913-1

BT - Proceedings

PB - Institute of Electrical and Electronic Engineers IEEE

ER -

Mämmelä O, Mannersalo P. Cognitive wireless access selection at client side: Performance study of a Q-learning approach. In Proceedings: IEEE Network Operations and Management Symposium, NOMS 2014. Institute of Electrical and Electronic Engineers IEEE. 2014. 6838343 https://doi.org/10.1109/NOMS.2014.6838343