MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer

Kaj Helin (Corresponding author), Jaakko Karjalainen, Paul Kiernan, Mikael Wolff, David Martinez Oliveira

Research output: Chapter in Book/Report/Conference proceedingConference abstract in proceedingsProfessional

Abstract

This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.
Original languageEnglish
Title of host publicationThe application track, posters and demos of EuroVR
Subtitle of host publicationProceedings of the 16th Annual EuroVR Conference 2019
PublisherVTT Technical Research Centre of Finland
Pages9-12
Number of pages128
ISBN (Electronic)978-951-38-8693-6
DOIs
Publication statusPublished - 2019
MoE publication typeD3 Professional conference proceedings
Event16th EuroVR International Conference—EuroVR 2019 - Tallin, Estonia
Duration: 23 Oct 201925 Oct 2019

Publication series

SeriesVTT Technology
Number357
ISSN2242-1211

Conference

Conference16th EuroVR International Conference—EuroVR 2019
CountryEstonia
CityTallin
Period23/10/1925/10/19

Fingerprint

User interfaces
Augmented reality
Smartphones
Space stations
Robotics
Remote control
Graphical user interfaces
NASA
Ultrasonics
Display devices
Experiments

Keywords

  • mixed reality
  • hands-free
  • MobiPV
  • ISS procedures

Cite this

Helin, K., Karjalainen, J., Kiernan, P., Wolff, M., & Oliveira, D. M. (2019). MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. In The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019 (pp. 9-12). VTT Technical Research Centre of Finland. VTT Technology, No. 357 https://doi.org/10.32040/2242-122X.2019.T357
Helin, Kaj ; Karjalainen, Jaakko ; Kiernan, Paul ; Wolff, Mikael ; Oliveira, David Martinez. / MobiPV4Hololens : HoloLens Based User Interface for Astronauts Procedure Viewer. The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, 2019. pp. 9-12 (VTT Technology; No. 357).
@inbook{68d6469c0f2e45f48f558a0c4cbab98b,
title = "MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer",
abstract = "This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.",
keywords = "mixed reality, hands-free, MobiPV, ISS procedures",
author = "Kaj Helin and Jaakko Karjalainen and Paul Kiernan and Mikael Wolff and Oliveira, {David Martinez}",
year = "2019",
doi = "10.32040/2242-122X.2019.T357",
language = "English",
series = "VTT Technology",
publisher = "VTT Technical Research Centre of Finland",
number = "357",
pages = "9--12",
booktitle = "The application track, posters and demos of EuroVR",
address = "Finland",

}

Helin, K, Karjalainen, J, Kiernan, P, Wolff, M & Oliveira, DM 2019, MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. in The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, VTT Technology, no. 357, pp. 9-12, 16th EuroVR International Conference—EuroVR 2019, Tallin, Estonia, 23/10/19. https://doi.org/10.32040/2242-122X.2019.T357

MobiPV4Hololens : HoloLens Based User Interface for Astronauts Procedure Viewer. / Helin, Kaj (Corresponding author); Karjalainen, Jaakko; Kiernan, Paul; Wolff, Mikael; Oliveira, David Martinez.

The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, 2019. p. 9-12 (VTT Technology; No. 357).

Research output: Chapter in Book/Report/Conference proceedingConference abstract in proceedingsProfessional

TY - CHAP

T1 - MobiPV4Hololens

T2 - HoloLens Based User Interface for Astronauts Procedure Viewer

AU - Helin, Kaj

AU - Karjalainen, Jaakko

AU - Kiernan, Paul

AU - Wolff, Mikael

AU - Oliveira, David Martinez

PY - 2019

Y1 - 2019

N2 - This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.

AB - This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.

KW - mixed reality

KW - hands-free

KW - MobiPV

KW - ISS procedures

U2 - 10.32040/2242-122X.2019.T357

DO - 10.32040/2242-122X.2019.T357

M3 - Conference abstract in proceedings

T3 - VTT Technology

SP - 9

EP - 12

BT - The application track, posters and demos of EuroVR

PB - VTT Technical Research Centre of Finland

ER -

Helin K, Karjalainen J, Kiernan P, Wolff M, Oliveira DM. MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. In The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland. 2019. p. 9-12. (VTT Technology; No. 357). https://doi.org/10.32040/2242-122X.2019.T357