E-Tree: Emotionally driven Augmented reality art

Stephen W. Gilroy, Marc Cavazza, Rémi Chaignon, Satu-Marja Mäkelä, Markus Niranen, Elisabeth André, Thurid Vogt, Mark Billinghurst, Harmut Seichter, Maurice Benayoun

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    27 Citations (Scopus)


    In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators.
    The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs.
    The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour.
    We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.
    Original languageEnglish
    Title of host publicationProceedings of the 16th ACM international conference on Multimedia, MM 2008
    PublisherAssociation for Computing Machinery ACM
    ISBN (Print)978-1-60558-303-7
    Publication statusPublished - 2008
    MoE publication typeA4 Article in a conference publication
    Event16th ACM international conference on Multimedia, MM 2008 - Vancouver, British Columbia, Canada
    Duration: 26 Oct 200831 Oct 2008


    Conference16th ACM international conference on Multimedia, MM 2008
    Abbreviated titleMM 2008
    CityVancouver, British Columbia


    • Affective computing
    • augmented reality
    • interactive digital arts
    • multimodal interaction


    Dive into the research topics of 'E-Tree: Emotionally driven Augmented reality art'. Together they form a unique fingerprint.

    Cite this