MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer

Kaj Helin (Corresponding author), Jaakko Karjalainen, Paul Kiernan, Mikael Wolff, David Martinez Oliveira

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsProfessional


    This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.
    Original languageEnglish
    Title of host publicationThe application track, posters and demos of EuroVR
    Subtitle of host publicationProceedings of the 16th Annual EuroVR Conference 2019
    EditorsKaj Helin, Jérôme Perret, Vladimir Kuts
    PublisherVTT Technical Research Centre of Finland
    ISBN (Electronic)978-951-38-8693-6
    Publication statusPublished - 2019
    MoE publication typeD3 Professional conference proceedings
    Event16th EuroVR International Conference—EuroVR 2019 - Tallin, Estonia
    Duration: 23 Oct 201925 Oct 2019

    Publication series

    SeriesVTT Technology


    Conference16th EuroVR International Conference—EuroVR 2019


    • mixed reality
    • hands-free
    • MobiPV
    • ISS procedures


    Dive into the research topics of 'MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer'. Together they form a unique fingerprint.

    Cite this