Semi-supervised context adaptation: Case study of audience excitement recognition

Elena Vildjiounaite (Corresponding Author), Vesa Kyllönen, Satu-Marja Mäkelä, Olli Vuorinen, Tommi Keränen, Johannes Peltola, G. Gimel'farb

    Research output: Contribution to journalArticleScientificpeer-review

    3 Citations (Scopus)

    Abstract

    To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.
    Original languageEnglish
    Pages (from-to)231-250
    Number of pages20
    JournalMultimedia Systems
    Volume18
    Issue number3
    DOIs
    Publication statusPublished - 2012
    MoE publication typeA1 Journal article-refereed

    Fingerprint

    Hidden Markov models
    Classifiers
    Sports
    Labeling
    Support vector machines
    Experiments

    Keywords

    • Audience responses
    • context adaptation
    • hidden Markov models
    • semi-supervised learning

    Cite this

    @article{be15ee1b54004f84bcb72a638710d43f,
    title = "Semi-supervised context adaptation: Case study of audience excitement recognition",
    abstract = "To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10{\%} higher accuracy than the conventional HMM and support vector machine based classifiers.",
    keywords = "Audience responses, context adaptation, hidden Markov models, semi-supervised learning",
    author = "Elena Vildjiounaite and Vesa Kyll{\"o}nen and Satu-Marja M{\"a}kel{\"a} and Olli Vuorinen and Tommi Ker{\"a}nen and Johannes Peltola and G. Gimel'farb",
    year = "2012",
    doi = "10.1007/s00530-012-0257-1",
    language = "English",
    volume = "18",
    pages = "231--250",
    journal = "Multimedia Systems",
    issn = "0942-4962",
    publisher = "Springer",
    number = "3",

    }

    Semi-supervised context adaptation : Case study of audience excitement recognition. / Vildjiounaite, Elena (Corresponding Author); Kyllönen, Vesa; Mäkelä, Satu-Marja; Vuorinen, Olli; Keränen, Tommi; Peltola, Johannes; Gimel'farb, G.

    In: Multimedia Systems, Vol. 18, No. 3, 2012, p. 231-250.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - Semi-supervised context adaptation

    T2 - Case study of audience excitement recognition

    AU - Vildjiounaite, Elena

    AU - Kyllönen, Vesa

    AU - Mäkelä, Satu-Marja

    AU - Vuorinen, Olli

    AU - Keränen, Tommi

    AU - Peltola, Johannes

    AU - Gimel'farb, G.

    PY - 2012

    Y1 - 2012

    N2 - To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

    AB - To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

    KW - Audience responses

    KW - context adaptation

    KW - hidden Markov models

    KW - semi-supervised learning

    U2 - 10.1007/s00530-012-0257-1

    DO - 10.1007/s00530-012-0257-1

    M3 - Article

    VL - 18

    SP - 231

    EP - 250

    JO - Multimedia Systems

    JF - Multimedia Systems

    SN - 0942-4962

    IS - 3

    ER -