Reinforcement learning method for QoE-aware optimization of content delivery

Faqir Zarrar Yousaf, Olli Mämmelä, Petteri Mannersalo

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

3 Citations (Scopus)

Abstract

The delivery of video services in a controllable and resource efficient manner while meeting the various QoE/QoS requirements in mobile networks is a challenging task, especially in a multiclass wireless environment. This paper proposes an intelligent and context-aware application level fair scheduler based on reinforcement-learning, which can dynamically adjust relevant scheduling parameters in reaction to specific events or context information. The implemented Q-learning method is analyzed with reference to the delivery of progressive video streaming services. We first highlight the performance issues during progressive video streaming over TCP to multiple users under resource constrained environment. We then demonstrate the utilization of employing Q-learning method in our scheduler for intelligent orchestration between multiple concurrent flows to ensure against buffer starvation and thus enable smooth playback. We also demonstrate the effectiveness of our context-aware dynamic scheduler to provide service separation between the user classes and fairness within a user class.
Original languageEnglish
Title of host publicationProceedings
Subtitle of host publicationIEEE Wireless Communications and Networking Conference, WCNC 2014
PublisherInstitute of Electrical and Electronic Engineers IEEE
Pages3390-3395
ISBN (Electronic)978-1-4799-3083-8
DOIs
Publication statusPublished - 2014
MoE publication typeA4 Article in a conference publication
EventWireless Communications and Networking Conference, WCNC 2014 - Istanbul, Turkey
Duration: 6 Apr 20149 Apr 2014

Publication series

NameConference proceedings, IEEE
PublisherIEEE

Conference

ConferenceWireless Communications and Networking Conference, WCNC 2014
Abbreviated titleWCNC 2014
CountryTurkey
CityIstanbul
Period6/04/149/04/14

Fingerprint

Video streaming
Reinforcement learning
Wireless networks
Quality of service
Scheduling

Keywords

  • dynamic scheduling
  • mobile computing
  • quality of experience
  • video streaming
  • Q-learning method
  • QoE-aware optimization
  • reinforcement learning method

Cite this

Yousaf, F. Z., Mämmelä, O., & Mannersalo, P. (2014). Reinforcement learning method for QoE-aware optimization of content delivery. In Proceedings: IEEE Wireless Communications and Networking Conference, WCNC 2014 (pp. 3390-3395). Institute of Electrical and Electronic Engineers IEEE. Conference proceedings, IEEE https://doi.org/10.1109/WCNC.2014.6953124
Yousaf, Faqir Zarrar ; Mämmelä, Olli ; Mannersalo, Petteri. / Reinforcement learning method for QoE-aware optimization of content delivery. Proceedings: IEEE Wireless Communications and Networking Conference, WCNC 2014. Institute of Electrical and Electronic Engineers IEEE, 2014. pp. 3390-3395 (Conference proceedings, IEEE).
@inproceedings{590709d396e540fc8c4c39d9e49c505e,
title = "Reinforcement learning method for QoE-aware optimization of content delivery",
abstract = "The delivery of video services in a controllable and resource efficient manner while meeting the various QoE/QoS requirements in mobile networks is a challenging task, especially in a multiclass wireless environment. This paper proposes an intelligent and context-aware application level fair scheduler based on reinforcement-learning, which can dynamically adjust relevant scheduling parameters in reaction to specific events or context information. The implemented Q-learning method is analyzed with reference to the delivery of progressive video streaming services. We first highlight the performance issues during progressive video streaming over TCP to multiple users under resource constrained environment. We then demonstrate the utilization of employing Q-learning method in our scheduler for intelligent orchestration between multiple concurrent flows to ensure against buffer starvation and thus enable smooth playback. We also demonstrate the effectiveness of our context-aware dynamic scheduler to provide service separation between the user classes and fairness within a user class.",
keywords = "dynamic scheduling, mobile computing, quality of experience, video streaming, Q-learning method, QoE-aware optimization, reinforcement learning method",
author = "Yousaf, {Faqir Zarrar} and Olli M{\"a}mmel{\"a} and Petteri Mannersalo",
note = "Project code: 82164",
year = "2014",
doi = "10.1109/WCNC.2014.6953124",
language = "English",
series = "Conference proceedings, IEEE",
publisher = "Institute of Electrical and Electronic Engineers IEEE",
pages = "3390--3395",
booktitle = "Proceedings",
address = "United States",

}

Yousaf, FZ, Mämmelä, O & Mannersalo, P 2014, Reinforcement learning method for QoE-aware optimization of content delivery. in Proceedings: IEEE Wireless Communications and Networking Conference, WCNC 2014. Institute of Electrical and Electronic Engineers IEEE, Conference proceedings, IEEE, pp. 3390-3395, Wireless Communications and Networking Conference, WCNC 2014, Istanbul, Turkey, 6/04/14. https://doi.org/10.1109/WCNC.2014.6953124

Reinforcement learning method for QoE-aware optimization of content delivery. / Yousaf, Faqir Zarrar; Mämmelä, Olli; Mannersalo, Petteri.

Proceedings: IEEE Wireless Communications and Networking Conference, WCNC 2014. Institute of Electrical and Electronic Engineers IEEE, 2014. p. 3390-3395 (Conference proceedings, IEEE).

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Reinforcement learning method for QoE-aware optimization of content delivery

AU - Yousaf, Faqir Zarrar

AU - Mämmelä, Olli

AU - Mannersalo, Petteri

N1 - Project code: 82164

PY - 2014

Y1 - 2014

N2 - The delivery of video services in a controllable and resource efficient manner while meeting the various QoE/QoS requirements in mobile networks is a challenging task, especially in a multiclass wireless environment. This paper proposes an intelligent and context-aware application level fair scheduler based on reinforcement-learning, which can dynamically adjust relevant scheduling parameters in reaction to specific events or context information. The implemented Q-learning method is analyzed with reference to the delivery of progressive video streaming services. We first highlight the performance issues during progressive video streaming over TCP to multiple users under resource constrained environment. We then demonstrate the utilization of employing Q-learning method in our scheduler for intelligent orchestration between multiple concurrent flows to ensure against buffer starvation and thus enable smooth playback. We also demonstrate the effectiveness of our context-aware dynamic scheduler to provide service separation between the user classes and fairness within a user class.

AB - The delivery of video services in a controllable and resource efficient manner while meeting the various QoE/QoS requirements in mobile networks is a challenging task, especially in a multiclass wireless environment. This paper proposes an intelligent and context-aware application level fair scheduler based on reinforcement-learning, which can dynamically adjust relevant scheduling parameters in reaction to specific events or context information. The implemented Q-learning method is analyzed with reference to the delivery of progressive video streaming services. We first highlight the performance issues during progressive video streaming over TCP to multiple users under resource constrained environment. We then demonstrate the utilization of employing Q-learning method in our scheduler for intelligent orchestration between multiple concurrent flows to ensure against buffer starvation and thus enable smooth playback. We also demonstrate the effectiveness of our context-aware dynamic scheduler to provide service separation between the user classes and fairness within a user class.

KW - dynamic scheduling

KW - mobile computing

KW - quality of experience

KW - video streaming

KW - Q-learning method

KW - QoE-aware optimization

KW - reinforcement learning method

U2 - 10.1109/WCNC.2014.6953124

DO - 10.1109/WCNC.2014.6953124

M3 - Conference article in proceedings

T3 - Conference proceedings, IEEE

SP - 3390

EP - 3395

BT - Proceedings

PB - Institute of Electrical and Electronic Engineers IEEE

ER -

Yousaf FZ, Mämmelä O, Mannersalo P. Reinforcement learning method for QoE-aware optimization of content delivery. In Proceedings: IEEE Wireless Communications and Networking Conference, WCNC 2014. Institute of Electrical and Electronic Engineers IEEE. 2014. p. 3390-3395. (Conference proceedings, IEEE). https://doi.org/10.1109/WCNC.2014.6953124