MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer

This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.