Context aware multimodal user interfaces for mobile work machines

Traditionally, work machines (road scrapers, excavation work machines, etc.) have been manually controlled with the aid of joysticks, switches, buttons and pedals. Automation of work machines is evolving slowly by embedding intelligent semi-automatic, automatic or remote control features to assist the operator in his tasks. Along with the time and growing degree of automation, various graphical user interfaces to show sensor and parameter readings to the operator have been exploited. The growing automation level is changing the role of the operator from the performance of task executions to the supervision of these operations. Hence, in the future, user interfaces (UI) will have to support these operations. Both adaptive context-aware technologies and multimodal solutions should be considered when developing user interface solutions for sophisticated work machines. An adaptive UI is context-sensitive - i.e. it will adapt to various situations during use. These include changes in the work (the current task status), the user (role, status and way of action), the environment and the status of the work machine. Multimodality introduces the natural way of interaction between human and work machines. Multimodal techniques include, e.g., force feedback, haptics, voice and gestures in addition to traditional WIMP (windows, icons, menus, push buttons) interfaces. In this paper we introduce a context-aware multimodal user interface for mobile work machines.