Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation

Mohammad Tavakolian (Corresponding Author), Miguel Bordallo Lopez, Li Liu

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Recently, automatic pain assessment technology, in particular automatically detecting pain from facial expressions, has been developed to improve the quality of pain management, and has attracted increasing attention. In this paper, we propose self-supervised learning for automatic yet efficient pain assessment, in order to reduce the cost of collecting large amount of labeled data. To achieve this, we introduce a novel similarity function to learn generalized representations using a Siamese network in the pretext task. The learned representations are finetuned in the downstream task of pain intensity estimation. To make the method computationally efficient, we propose Statistical Spatiotemporal Distillation (SSD) to encode the spatiotemporal variations underlying the facial video into a single RGB image, enabling the use of less complex 2D deep models for video representation. Experiments on two publicly available pain datasets and cross-dataset evaluation demonstrate promising results, showing the good generalization ability of the learned representations.

Original languageEnglish
Pages (from-to)26-33
Number of pages8
JournalPattern Recognition Letters
Volume140
DOIs
Publication statusPublished - Dec 2020
MoE publication typeA1 Journal article-refereed

Keywords

  • Pain assessment
  • Representation learning
  • Self-supervised learning
  • Statistical spatiotemporal distillation

Fingerprint Dive into the research topics of 'Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation'. Together they form a unique fingerprint.

  • Cite this