MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer

Kaj Helin (Corresponding author), Jaakko Karjalainen, Paul Kiernan, Mikael Wolff, David Martinez Oliveira

    Research output: Chapter in Book/Report/Conference proceedingConference abstract in proceedingsProfessional

    Abstract

    This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.
    Original languageEnglish
    Title of host publicationThe application track, posters and demos of EuroVR
    Subtitle of host publicationProceedings of the 16th Annual EuroVR Conference 2019
    PublisherVTT Technical Research Centre of Finland
    Pages9-12
    Number of pages128
    ISBN (Electronic)978-951-38-8693-6
    DOIs
    Publication statusPublished - 2019
    MoE publication typeD3 Professional conference proceedings
    Event16th EuroVR International Conference—EuroVR 2019 - Tallin, Estonia
    Duration: 23 Oct 201925 Oct 2019

    Publication series

    SeriesVTT Technology
    Number357
    ISSN2242-1211

    Conference

    Conference16th EuroVR International Conference—EuroVR 2019
    CountryEstonia
    CityTallin
    Period23/10/1925/10/19

    Fingerprint

    User interfaces
    Augmented reality
    Smartphones
    Space stations
    Robotics
    Remote control
    Graphical user interfaces
    NASA
    Ultrasonics
    Display devices
    Experiments

    Keywords

    • mixed reality
    • hands-free
    • MobiPV
    • ISS procedures

    Cite this

    Helin, K., Karjalainen, J., Kiernan, P., Wolff, M., & Oliveira, D. M. (2019). MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. In The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019 (pp. 9-12). VTT Technical Research Centre of Finland. VTT Technology, No. 357 https://doi.org/10.32040/2242-122X.2019.T357
    Helin, Kaj ; Karjalainen, Jaakko ; Kiernan, Paul ; Wolff, Mikael ; Oliveira, David Martinez. / MobiPV4Hololens : HoloLens Based User Interface for Astronauts Procedure Viewer. The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, 2019. pp. 9-12 (VTT Technology; No. 357).
    @inbook{68d6469c0f2e45f48f558a0c4cbab98b,
    title = "MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer",
    abstract = "This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.",
    keywords = "mixed reality, hands-free, MobiPV, ISS procedures",
    author = "Kaj Helin and Jaakko Karjalainen and Paul Kiernan and Mikael Wolff and Oliveira, {David Martinez}",
    year = "2019",
    doi = "10.32040/2242-122X.2019.T357",
    language = "English",
    series = "VTT Technology",
    publisher = "VTT Technical Research Centre of Finland",
    number = "357",
    pages = "9--12",
    booktitle = "The application track, posters and demos of EuroVR",
    address = "Finland",

    }

    Helin, K, Karjalainen, J, Kiernan, P, Wolff, M & Oliveira, DM 2019, MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. in The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, VTT Technology, no. 357, pp. 9-12, 16th EuroVR International Conference—EuroVR 2019, Tallin, Estonia, 23/10/19. https://doi.org/10.32040/2242-122X.2019.T357

    MobiPV4Hololens : HoloLens Based User Interface for Astronauts Procedure Viewer. / Helin, Kaj (Corresponding author); Karjalainen, Jaakko; Kiernan, Paul; Wolff, Mikael; Oliveira, David Martinez.

    The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland, 2019. p. 9-12 (VTT Technology; No. 357).

    Research output: Chapter in Book/Report/Conference proceedingConference abstract in proceedingsProfessional

    TY - CHAP

    T1 - MobiPV4Hololens

    T2 - HoloLens Based User Interface for Astronauts Procedure Viewer

    AU - Helin, Kaj

    AU - Karjalainen, Jaakko

    AU - Kiernan, Paul

    AU - Wolff, Mikael

    AU - Oliveira, David Martinez

    PY - 2019

    Y1 - 2019

    N2 - This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.

    AB - This application abstract introduces a Proof-of-Concept (PoC) for Mixed Reality (MR) system to support an astronaut’s manual work, the system is called MobiPV4Hololens. It has been developed in the European Space Agency’s (ESA) project called “MobiPV4Hololens - Prototype a Media Helmet for MobiPV Implemented Using Microsoft (MS) HoloLens”. The MS HoloLens mixed reality platform (Microsoft, 2019) was integrated as the hands-free user interface to the ESA Mobile Procedure Viewer system called MobiPV. The International Space Station (ISS) tasks can range from laptop usage procedures, robotic teleoperation, biology experiments to ultrasound diagnosis. These manual procedures are based on standard task description schemas such as the Operations Data File (ODF) (NASA, 2010). During normal procedure execution, an astronaut would have a laptop nearby with step-by-step instructions to follow. The ESA’s MobiPV (Boyd et al., 2016) solution supports astronauts through wearing a smartphone on their wrist that connects to the ODF library and shows the instructions on-screen. MR and/or Augmented Reality (AR) has been tested within several projects of the space domain in the fields of training and manual work support (Tedone et al., 2016). and supporting robotics operation in the ISS (Maida et al., 2007). The MR and/or AR usability has reach acceptable level in space related training and maintenance support (Helin et al., 2018). The objective of the activity was to complement the MobiPV astronaut smartphone with a wireless media helmet in this case MS HoloLens. The software implemented a Natural User Interface (NUI), i.e. a conventional graphical user interface (GUI) on the HoloLens head-up display, complemented with speech input for user interface commanding, and speech output for procedure readback.

    KW - mixed reality

    KW - hands-free

    KW - MobiPV

    KW - ISS procedures

    U2 - 10.32040/2242-122X.2019.T357

    DO - 10.32040/2242-122X.2019.T357

    M3 - Conference abstract in proceedings

    T3 - VTT Technology

    SP - 9

    EP - 12

    BT - The application track, posters and demos of EuroVR

    PB - VTT Technical Research Centre of Finland

    ER -

    Helin K, Karjalainen J, Kiernan P, Wolff M, Oliveira DM. MobiPV4Hololens: HoloLens Based User Interface for Astronauts Procedure Viewer. In The application track, posters and demos of EuroVR: Proceedings of the 16th Annual EuroVR Conference 2019. VTT Technical Research Centre of Finland. 2019. p. 9-12. (VTT Technology; No. 357). https://doi.org/10.32040/2242-122X.2019.T357