Semi-supervised context adaptation: Case study of audience excitement recognition

Elena Vildjiounaite (Corresponding Author), Vesa Kyllönen, Satu-Marja Mäkelä, Olli Vuorinen, Tommi Keränen, Johannes Peltola, G. Gimel'farb

Research output: Contribution to journalArticleScientificpeer-review

3 Citations (Scopus)

Abstract

To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.
Original languageEnglish
Pages (from-to)231-250
Number of pages20
JournalMultimedia Systems
Volume18
Issue number3
DOIs
Publication statusPublished - 2012
MoE publication typeA1 Journal article-refereed

Fingerprint

Hidden Markov models
Classifiers
Sports
Labeling
Support vector machines
Experiments

Keywords

  • Audience responses
  • context adaptation
  • hidden Markov models
  • semi-supervised learning

Cite this

@article{be15ee1b54004f84bcb72a638710d43f,
title = "Semi-supervised context adaptation: Case study of audience excitement recognition",
abstract = "To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10{\%} higher accuracy than the conventional HMM and support vector machine based classifiers.",
keywords = "Audience responses, context adaptation, hidden Markov models, semi-supervised learning",
author = "Elena Vildjiounaite and Vesa Kyll{\"o}nen and Satu-Marja M{\"a}kel{\"a} and Olli Vuorinen and Tommi Ker{\"a}nen and Johannes Peltola and G. Gimel'farb",
year = "2012",
doi = "10.1007/s00530-012-0257-1",
language = "English",
volume = "18",
pages = "231--250",
journal = "Multimedia Systems",
issn = "0942-4962",
publisher = "Springer",
number = "3",

}

Semi-supervised context adaptation : Case study of audience excitement recognition. / Vildjiounaite, Elena (Corresponding Author); Kyllönen, Vesa; Mäkelä, Satu-Marja; Vuorinen, Olli; Keränen, Tommi; Peltola, Johannes; Gimel'farb, G.

In: Multimedia Systems, Vol. 18, No. 3, 2012, p. 231-250.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Semi-supervised context adaptation

T2 - Case study of audience excitement recognition

AU - Vildjiounaite, Elena

AU - Kyllönen, Vesa

AU - Mäkelä, Satu-Marja

AU - Vuorinen, Olli

AU - Keränen, Tommi

AU - Peltola, Johannes

AU - Gimel'farb, G.

PY - 2012

Y1 - 2012

N2 - To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

AB - To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter—in contrast to the context-specific one—adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user’s annotations, the proposed semi-supervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user- and context-adaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

KW - Audience responses

KW - context adaptation

KW - hidden Markov models

KW - semi-supervised learning

U2 - 10.1007/s00530-012-0257-1

DO - 10.1007/s00530-012-0257-1

M3 - Article

VL - 18

SP - 231

EP - 250

JO - Multimedia Systems

JF - Multimedia Systems

SN - 0942-4962

IS - 3

ER -