Video2IMU: Realistic IMU features and signals from videos

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

Human Activity Recognition (HAR) from wearable sensor data identifies movements or activities in unconstrained environments. HAR is a challenging problem as it presents great variability across subjects. Obtaining large amounts of labelled data is not straightforward, since wearable sensor signals are not easy to label upon simple human inspection. In our work, we propose the use of neural networks for the generation of realistic signals and features using human activity monocular videos. We show how these generated features and signals can be utilized, instead of their real counterparts, to train HAR models that can recognize activities using signals obtained with wearable sensors. To prove the validity of our methods, we perform experiments on an activity recognition dataset created for the improvement of industrial work safety. We show that our model is able to realistically generate virtual sensor signals and features usable to train a HAR classifier with comparable performance as the one trained using real sensor data. Our results enable the use of available, labeled video data for training HAR models to classify signals from wearable sensors.
Original languageEnglish
Title of host publication2022 IEEE-EMBS International Conference on Biomedical and Health Informatics and IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks
Subtitle of host publicationProceedings
PublisherIEEE Institute of Electrical and Electronic Engineers
Number of pages5
ISBN (Electronic)978-1-6654-5925-9
ISBN (Print)978-1-6654-5926-6
DOIs
Publication statusPublished - 1 Nov 2022
MoE publication typeA4 Article in a conference publication
EventIEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks, BSN 2022 - Ioannina, Greece
Duration: 27 Sep 202230 Sep 2022

Publication series

SeriesInternational Workshop on Wearable and Implantable Body Sensor Networks (BSN)

Conference

ConferenceIEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks, BSN 2022
Country/TerritoryGreece
CityIoannina
Period27/09/2230/09/22

Keywords

  • activity recognition
  • IMU
  • multimodal representation
  • signal transformation

Fingerprint

Dive into the research topics of 'Video2IMU: Realistic IMU features and signals from videos'. Together they form a unique fingerprint.

Cite this