Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications

Ky Trung Nguyen; Jani Mäntyjärvi; Tran Thi Ngoc Nguyen

doi:10.1109/BHI50953.2021.9508616

Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications

Ky Trung Nguyen, Jani Mäntyjärvi, Tran Thi Ngoc Nguyen

BA63 Sensing solutions

Industrial University of Ho Chi Minh City

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

Storytelling describes our daily living activities in many ways. It assists us to understand what we have done to advance our health and wellbeing. In this paper, we present our novel approach to generate scripts from events, which are detected from wearable sensor data. First, we use Deep Neural Network (DNN) to recognize semantic concepts such as gesture, activity, and location for generating a chronological sequence of events. Second, we apply a sequence to sequence (SEQ2SEQ) model consisting of two recurrent neural networks (RNNs) to generate human-understandable stories. The results show that our method can improve the performance of script generation (SG) by using SEQ2SEQ with 0.972 BLEU-1 score.

Original language	English
Title of host publication	BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings
Publisher	IEEE Institute of Electrical and Electronic Engineers
Pages	1-4
Number of pages	4
ISBN (Electronic)	978-1-6654-0358-0
ISBN (Print)	978-1-6654-4770-6
DOIs	https://doi.org/10.1109/BHI50953.2021.9508616
Publication status	Published - 30 Jul 2021
MoE publication type	A4 Article in a conference publication
Event	IEEE EMBS International Conference on Biomedical and Health Informatics, BHI: Online - Virtual, Athens, Greece Duration: 27 Jul 2021 → 30 Jul 2021

Conference

Conference	IEEE EMBS International Conference on Biomedical and Health Informatics, BHI
Abbreviated title	IEEE BHI-BSN 2021
Country/Territory	Greece
City	Athens
Period	27/07/21 → 30/07/21

Keywords

Storytelling
Human Activity Recognition
Machine Learning
Script Generation

Access to Document

10.1109/BHI50953.2021.9508616

Cite this

Nguyen, K. T., Mäntyjärvi, J., & Nguyen, T. T. N. (2021). Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications. In BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings (pp. 1-4). Article 9508616 IEEE Institute of Electrical and Electronic Engineers. https://doi.org/10.1109/BHI50953.2021.9508616

@inproceedings{bf4a39ec8af246028a82717a6ee5b3d4,

title = "Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications",

abstract = "Storytelling describes our daily living activities in many ways. It assists us to understand what we have done to advance our health and wellbeing. In this paper, we present our novel approach to generate scripts from events, which are detected from wearable sensor data. First, we use Deep Neural Network (DNN) to recognize semantic concepts such as gesture, activity, and location for generating a chronological sequence of events. Second, we apply a sequence to sequence (SEQ2SEQ) model consisting of two recurrent neural networks (RNNs) to generate human-understandable stories. The results show that our method can improve the performance of script generation (SG) by using SEQ2SEQ with 0.972 BLEU-1 score.",

keywords = "Storytelling, Human Activity Recognition, Machine Learning, Script Generation",

author = "Nguyen, {Ky Trung} and Jani M{\"a}ntyj{\"a}rvi and Nguyen, {Tran Thi Ngoc}",

note = "Funding Information: This work was carried out during the tenure of an ERCIM'Alain Bensoussan' Fellowship Programme. Publisher Copyright: {\textcopyright} 2021 IEEE; IEEE EMBS International Conference on Biomedical and Health Informatics, BHI : Online, IEEE BHI-BSN 2021 ; Conference date: 27-07-2021 Through 30-07-2021",

year = "2021",

month = jul,

day = "30",

doi = "10.1109/BHI50953.2021.9508616",

language = "English",

isbn = "978-1-6654-4770-6",

pages = "1--4",

booktitle = "BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings",

publisher = "IEEE Institute of Electrical and Electronic Engineers",

address = "United States",

}

Nguyen, KT, Mäntyjärvi, J & Nguyen, TTN 2021, Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications. in BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings., 9508616, IEEE Institute of Electrical and Electronic Engineers, pp. 1-4, IEEE EMBS International Conference on Biomedical and Health Informatics, BHI, Athens, Greece, 27/07/21. https://doi.org/10.1109/BHI50953.2021.9508616

Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications. / Nguyen, Ky Trung; Mäntyjärvi, Jani; Nguyen, Tran Thi Ngoc.
BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings. IEEE Institute of Electrical and Electronic Engineers, 2021. p. 1-4 9508616.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications

AU - Nguyen, Ky Trung

AU - Mäntyjärvi, Jani

AU - Nguyen, Tran Thi Ngoc

PY - 2021/7/30

Y1 - 2021/7/30

N2 - Storytelling describes our daily living activities in many ways. It assists us to understand what we have done to advance our health and wellbeing. In this paper, we present our novel approach to generate scripts from events, which are detected from wearable sensor data. First, we use Deep Neural Network (DNN) to recognize semantic concepts such as gesture, activity, and location for generating a chronological sequence of events. Second, we apply a sequence to sequence (SEQ2SEQ) model consisting of two recurrent neural networks (RNNs) to generate human-understandable stories. The results show that our method can improve the performance of script generation (SG) by using SEQ2SEQ with 0.972 BLEU-1 score.

AB - Storytelling describes our daily living activities in many ways. It assists us to understand what we have done to advance our health and wellbeing. In this paper, we present our novel approach to generate scripts from events, which are detected from wearable sensor data. First, we use Deep Neural Network (DNN) to recognize semantic concepts such as gesture, activity, and location for generating a chronological sequence of events. Second, we apply a sequence to sequence (SEQ2SEQ) model consisting of two recurrent neural networks (RNNs) to generate human-understandable stories. The results show that our method can improve the performance of script generation (SG) by using SEQ2SEQ with 0.972 BLEU-1 score.

KW - Storytelling

KW - Human Activity Recognition

KW - Machine Learning

KW - Script Generation

UR - http://www.scopus.com/inward/record.url?scp=85125447344&partnerID=8YFLogxK

U2 - 10.1109/BHI50953.2021.9508616

DO - 10.1109/BHI50953.2021.9508616

M3 - Conference article in proceedings

SN - 978-1-6654-4770-6

SP - 1

EP - 4

BT - BHI 2021 - 2021 IEEE EMBS International Conference on Biomedical and Health Informatics, Proceedings

PB - IEEE Institute of Electrical and Electronic Engineers

T2 - IEEE EMBS International Conference on Biomedical and Health Informatics, BHI

Y2 - 27 July 2021 through 30 July 2021

ER -

Automatic Storytelling from Wearable Sensor Data for Health and Wellness Applications

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this