E-Tree: Emotionally driven Augmented reality art

Stephen W. Gilroy, Marc Cavazza, Rémi Chaignon, Satu-Marja Mäkelä, Markus Niranen, Elisabeth André, Thurid Vogt, Mark Billinghurst, Harmut Seichter, Maurice Benayoun

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

20 Citations (Scopus)

Abstract

In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators.
The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs.
The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour.
We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.
Original languageEnglish
Title of host publicationProceedings of the 16th ACM international conference on Multimedia, MM 2008
PublisherAssociation for Computing Machinery ACM
Pages945-948
ISBN (Print)978-1-60558-303-7
DOIs
Publication statusPublished - 2008
MoE publication typeA4 Article in a conference publication
Event16th ACM international conference on Multimedia, MM 2008 - Vancouver, British Columbia, Canada
Duration: 26 Oct 200831 Oct 2008

Conference

Conference16th ACM international conference on Multimedia, MM 2008
Abbreviated titleMM 2008
CountryCanada
CityVancouver, British Columbia
Period26/10/0831/10/08

Fingerprint

Augmented reality
Projection systems
Mirrors
Fusion reactions
Display devices
Processing

Keywords

  • Affective computing
  • augmented reality
  • interactive digital arts
  • multimodal interaction

Cite this

Gilroy, S. W., Cavazza, M., Chaignon, R., Mäkelä, S-M., Niranen, M., André, E., ... Benayoun, M. (2008). E-Tree: Emotionally driven Augmented reality art. In Proceedings of the 16th ACM international conference on Multimedia, MM 2008 (pp. 945-948). Association for Computing Machinery ACM. https://doi.org/10.1145/1459359.1459529
Gilroy, Stephen W. ; Cavazza, Marc ; Chaignon, Rémi ; Mäkelä, Satu-Marja ; Niranen, Markus ; André, Elisabeth ; Vogt, Thurid ; Billinghurst, Mark ; Seichter, Harmut ; Benayoun, Maurice. / E-Tree : Emotionally driven Augmented reality art. Proceedings of the 16th ACM international conference on Multimedia, MM 2008. Association for Computing Machinery ACM, 2008. pp. 945-948
@inproceedings{e516c901578e4e7597b5c4351201eb48,
title = "E-Tree: Emotionally driven Augmented reality art",
abstract = "In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators. The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs. The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour.We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.",
keywords = "Affective computing, augmented reality, interactive digital arts, multimodal interaction",
author = "Gilroy, {Stephen W.} and Marc Cavazza and R{\'e}mi Chaignon and Satu-Marja M{\"a}kel{\"a} and Markus Niranen and Elisabeth Andr{\'e} and Thurid Vogt and Mark Billinghurst and Harmut Seichter and Maurice Benayoun",
year = "2008",
doi = "10.1145/1459359.1459529",
language = "English",
isbn = "978-1-60558-303-7",
pages = "945--948",
booktitle = "Proceedings of the 16th ACM international conference on Multimedia, MM 2008",
publisher = "Association for Computing Machinery ACM",
address = "United States",

}

Gilroy, SW, Cavazza, M, Chaignon, R, Mäkelä, S-M, Niranen, M, André, E, Vogt, T, Billinghurst, M, Seichter, H & Benayoun, M 2008, E-Tree: Emotionally driven Augmented reality art. in Proceedings of the 16th ACM international conference on Multimedia, MM 2008. Association for Computing Machinery ACM, pp. 945-948, 16th ACM international conference on Multimedia, MM 2008, Vancouver, British Columbia, Canada, 26/10/08. https://doi.org/10.1145/1459359.1459529

E-Tree : Emotionally driven Augmented reality art. / Gilroy, Stephen W.; Cavazza, Marc; Chaignon, Rémi; Mäkelä, Satu-Marja; Niranen, Markus; André, Elisabeth; Vogt, Thurid; Billinghurst, Mark; Seichter, Harmut; Benayoun, Maurice.

Proceedings of the 16th ACM international conference on Multimedia, MM 2008. Association for Computing Machinery ACM, 2008. p. 945-948.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - E-Tree

T2 - Emotionally driven Augmented reality art

AU - Gilroy, Stephen W.

AU - Cavazza, Marc

AU - Chaignon, Rémi

AU - Mäkelä, Satu-Marja

AU - Niranen, Markus

AU - André, Elisabeth

AU - Vogt, Thurid

AU - Billinghurst, Mark

AU - Seichter, Harmut

AU - Benayoun, Maurice

PY - 2008

Y1 - 2008

N2 - In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators. The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs. The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour.We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.

AB - In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators. The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs. The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour.We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.

KW - Affective computing

KW - augmented reality

KW - interactive digital arts

KW - multimodal interaction

U2 - 10.1145/1459359.1459529

DO - 10.1145/1459359.1459529

M3 - Conference article in proceedings

SN - 978-1-60558-303-7

SP - 945

EP - 948

BT - Proceedings of the 16th ACM international conference on Multimedia, MM 2008

PB - Association for Computing Machinery ACM

ER -

Gilroy SW, Cavazza M, Chaignon R, Mäkelä S-M, Niranen M, André E et al. E-Tree: Emotionally driven Augmented reality art. In Proceedings of the 16th ACM international conference on Multimedia, MM 2008. Association for Computing Machinery ACM. 2008. p. 945-948 https://doi.org/10.1145/1459359.1459529