Incorporating tilt-based interaction in multimodal user interfaces for mobile devices

Jani Mäntyjärvi, Fabio Paternò, Carmen Santoro

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

2 Citations (Scopus)

Abstract

Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for managing the increasing complexity consequent to the many available interaction platforms. However, they have usually focused on graphical and/or vocal modalities. This paper presents a solution for enabling the development of tilt-based hand gesture and graphical modalities for mobile devices in a multimodal user interface development tool. The challenges related to developing gesture-based applications for various types of devices involving mobile devices are discussed in detail. The possible solution presented is based on a logical description language for handgesture user interfaces. Such language allows us to obtain a user interface implementation on the target mobile platform. The solution is illustrated with an example application that can be accessed from both the desktop and mobile device supporting tiltbased gesture interaction
Original languageEnglish
Title of host publicationTask Models and Diagrams for Users Interface Design
EditorsK. Coninx, K. Luyten, K.A. Schneider
PublisherSpringer
Pages230-244
ISBN (Electronic)978-3-540-70816-2
ISBN (Print)978-3-540-70815-5
DOIs
Publication statusPublished - 2006
MoE publication typeA4 Article in a conference publication
Event5th International Workshop on TAsk MOdels and DIAgrams for user interface design, TAMODIA 2006 - Hasselt, Belgium
Duration: 23 Oct 200624 Oct 2006

Publication series

SeriesLecture Notes in Computer Science
Volume4385
ISSN0302-9743

Conference

Conference5th International Workshop on TAsk MOdels and DIAgrams for user interface design, TAMODIA 2006
Abbreviated titleTAMODIA 2006
CountryBelgium
CityHasselt
Period23/10/0624/10/06

Fingerprint

Mobile devices
User interfaces

Keywords

  • user interface
  • user interaction
  • human-computer interaction
  • model-based design
  • gestural interfaces
  • mobile device
  • tilt interface
  • tilt interaction
  • gesture interface
  • gesture interaction
  • accelerometer

Cite this

Mäntyjärvi, J., Paternò, F., & Santoro, C. (2006). Incorporating tilt-based interaction in multimodal user interfaces for mobile devices. In K. Coninx, K. Luyten, & K. A. Schneider (Eds.), Task Models and Diagrams for Users Interface Design (pp. 230-244). Springer. Lecture Notes in Computer Science, Vol.. 4385 https://doi.org/10.1007/978-3-540-70816-2_17
Mäntyjärvi, Jani ; Paternò, Fabio ; Santoro, Carmen. / Incorporating tilt-based interaction in multimodal user interfaces for mobile devices. Task Models and Diagrams for Users Interface Design. editor / K. Coninx ; K. Luyten ; K.A. Schneider. Springer, 2006. pp. 230-244 (Lecture Notes in Computer Science, Vol. 4385).
@inproceedings{efb617cd34064cf3891c45a6a4e196a0,
title = "Incorporating tilt-based interaction in multimodal user interfaces for mobile devices",
abstract = "Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for managing the increasing complexity consequent to the many available interaction platforms. However, they have usually focused on graphical and/or vocal modalities. This paper presents a solution for enabling the development of tilt-based hand gesture and graphical modalities for mobile devices in a multimodal user interface development tool. The challenges related to developing gesture-based applications for various types of devices involving mobile devices are discussed in detail. The possible solution presented is based on a logical description language for handgesture user interfaces. Such language allows us to obtain a user interface implementation on the target mobile platform. The solution is illustrated with an example application that can be accessed from both the desktop and mobile device supporting tiltbased gesture interaction",
keywords = "user interface, user interaction, human-computer interaction, model-based design, gestural interfaces, mobile device, tilt interface, tilt interaction, gesture interface, gesture interaction, accelerometer",
author = "Jani M{\"a}ntyj{\"a}rvi and Fabio Patern{\`o} and Carmen Santoro",
year = "2006",
doi = "10.1007/978-3-540-70816-2_17",
language = "English",
isbn = "978-3-540-70815-5",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "230--244",
editor = "K. Coninx and K. Luyten and K.A. Schneider",
booktitle = "Task Models and Diagrams for Users Interface Design",
address = "Germany",

}

Mäntyjärvi, J, Paternò, F & Santoro, C 2006, Incorporating tilt-based interaction in multimodal user interfaces for mobile devices. in K Coninx, K Luyten & KA Schneider (eds), Task Models and Diagrams for Users Interface Design. Springer, Lecture Notes in Computer Science, vol. 4385, pp. 230-244, 5th International Workshop on TAsk MOdels and DIAgrams for user interface design, TAMODIA 2006, Hasselt, Belgium, 23/10/06. https://doi.org/10.1007/978-3-540-70816-2_17

Incorporating tilt-based interaction in multimodal user interfaces for mobile devices. / Mäntyjärvi, Jani; Paternò, Fabio; Santoro, Carmen.

Task Models and Diagrams for Users Interface Design. ed. / K. Coninx; K. Luyten; K.A. Schneider. Springer, 2006. p. 230-244 (Lecture Notes in Computer Science, Vol. 4385).

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Incorporating tilt-based interaction in multimodal user interfaces for mobile devices

AU - Mäntyjärvi, Jani

AU - Paternò, Fabio

AU - Santoro, Carmen

PY - 2006

Y1 - 2006

N2 - Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for managing the increasing complexity consequent to the many available interaction platforms. However, they have usually focused on graphical and/or vocal modalities. This paper presents a solution for enabling the development of tilt-based hand gesture and graphical modalities for mobile devices in a multimodal user interface development tool. The challenges related to developing gesture-based applications for various types of devices involving mobile devices are discussed in detail. The possible solution presented is based on a logical description language for handgesture user interfaces. Such language allows us to obtain a user interface implementation on the target mobile platform. The solution is illustrated with an example application that can be accessed from both the desktop and mobile device supporting tiltbased gesture interaction

AB - Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for managing the increasing complexity consequent to the many available interaction platforms. However, they have usually focused on graphical and/or vocal modalities. This paper presents a solution for enabling the development of tilt-based hand gesture and graphical modalities for mobile devices in a multimodal user interface development tool. The challenges related to developing gesture-based applications for various types of devices involving mobile devices are discussed in detail. The possible solution presented is based on a logical description language for handgesture user interfaces. Such language allows us to obtain a user interface implementation on the target mobile platform. The solution is illustrated with an example application that can be accessed from both the desktop and mobile device supporting tiltbased gesture interaction

KW - user interface

KW - user interaction

KW - human-computer interaction

KW - model-based design

KW - gestural interfaces

KW - mobile device

KW - tilt interface

KW - tilt interaction

KW - gesture interface

KW - gesture interaction

KW - accelerometer

U2 - 10.1007/978-3-540-70816-2_17

DO - 10.1007/978-3-540-70816-2_17

M3 - Conference article in proceedings

SN - 978-3-540-70815-5

T3 - Lecture Notes in Computer Science

SP - 230

EP - 244

BT - Task Models and Diagrams for Users Interface Design

A2 - Coninx, K.

A2 - Luyten, K.

A2 - Schneider, K.A.

PB - Springer

ER -

Mäntyjärvi J, Paternò F, Santoro C. Incorporating tilt-based interaction in multimodal user interfaces for mobile devices. In Coninx K, Luyten K, Schneider KA, editors, Task Models and Diagrams for Users Interface Design. Springer. 2006. p. 230-244. (Lecture Notes in Computer Science, Vol. 4385). https://doi.org/10.1007/978-3-540-70816-2_17