MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer

Kaj Helin (Corresponding author), Jaakko Karjalainen, Paul Kiernan, Mikael Wolff, David Martinez Oliveira

Research output: Chapter in Book/Report/Conference proceedingConference abstract in proceedingsProfessional

Abstract

This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.
Original languageEnglish
Title of host publicationThe application track, posters and demos of EuroVR
Subtitle of host publicationProceedings of the 16th Annual EuroVR Conference 2019
PublisherVTT Technical Research Centre of Finland
Pages9-12
Number of pages128
ISBN (Electronic)978-951-38-8693-6
DOIs
Publication statusPublished - 2019
MoE publication typeD3 Professional conference proceedings
Event16th EuroVR International Conference—EuroVR 2019 - Tallin, Estonia
Duration: 23 Oct 201925 Oct 2019

Publication series

SeriesVTT Technology
Number357
ISSN2242-1211

Conference

Conference16th EuroVR International Conference—EuroVR 2019
CountryEstonia
CityTallin
Period23/10/1925/10/19

Keywords

  • mixed reality
  • hands-free
  • MobiPV
  • ISS procedures

Fingerprint Dive into the research topics of 'MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer'. Together they form a unique fingerprint.

  • Cite this

    Helin, K., Karjalainen, J., Kiernan, P., Wolff, M., & Oliveira, D. M. (2019). MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. In The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019 (pp. 9-12). VTT Technical Research Centre of Finland. VTT Technology, No. 357 https://doi.org/10.32040/2242-122X.2019.T357