Proficiency testing for sensory ranking panels: Measuring panel performance

Jean A. McEwan (Corresponding Author), Raija-Liisa Heiniö, E. Anthony Hunter, Per Lea

Research output: Contribution to journalArticleScientificpeer-review

13 Citations (Scopus)

Abstract

Proficiency testing in sensory analysis is an important step towards demonstrating that results from one sensory panel are consistent with the results of other sensory panels. The uniqueness of sensory analysis poses some specific problems for measuring the proficiency of the human instrument (panel). As part of an EU supported project, ProfiSens, 14 panels undertook ranking of sweetness on five samples of apple juice. Four panels were designated ‘validation’ panels, whose data were used firstly to establish the expected ranking results, and secondly to set the performance criteria that a trained sensory panel would be expected to achieve. Four key measures of a panel’s performance were investigated: the ability to rank the samples in the correct order; the significance level associated with differences between samples; the number of pairs of samples that a panel found to be different at a specified level of significance; and the degree of agreement between assessors within a panel. For each of these criteria an ‘expected result’ was considered, as well as an overall measure of performance. The data from the remaining panels were analysed, and the level of performance was recorded for each of the stated criteria. Results indicated different levels of performance, and the research also revealed the importance of choice of validation panels and the screening of samples prior to testing. A simpler performance scheme was proposed to address issues relating to attaching arbitrary weightings to each of the performance criteria, and to address potential problems associated with combining different measurement criteria into a single performance score. While there is still potential for further development and refinement, ProfiSens has made a significant contribution to proficiency testing for sensory analysis.
Original languageEnglish
Pages (from-to)247-256
Number of pages10
JournalFood Quality and Preference
Volume14
Issue number3
DOIs
Publication statusPublished - 2003
MoE publication typeA1 Journal article-refereed

Fingerprint

Malus
sensory evaluation
Research
testing
sampling
apple juice
sweetness
screening

Keywords

  • Proficiency testing
  • Panel performance
  • Ranking
  • Expected results

Cite this

McEwan, Jean A. ; Heiniö, Raija-Liisa ; Hunter, E. Anthony ; Lea, Per. / Proficiency testing for sensory ranking panels : Measuring panel performance. In: Food Quality and Preference. 2003 ; Vol. 14, No. 3. pp. 247-256.
@article{2f99603598f04063ba0aa84293f24ddf,
title = "Proficiency testing for sensory ranking panels: Measuring panel performance",
abstract = "Proficiency testing in sensory analysis is an important step towards demonstrating that results from one sensory panel are consistent with the results of other sensory panels. The uniqueness of sensory analysis poses some specific problems for measuring the proficiency of the human instrument (panel). As part of an EU supported project, ProfiSens, 14 panels undertook ranking of sweetness on five samples of apple juice. Four panels were designated ‘validation’ panels, whose data were used firstly to establish the expected ranking results, and secondly to set the performance criteria that a trained sensory panel would be expected to achieve. Four key measures of a panel’s performance were investigated: the ability to rank the samples in the correct order; the significance level associated with differences between samples; the number of pairs of samples that a panel found to be different at a specified level of significance; and the degree of agreement between assessors within a panel. For each of these criteria an ‘expected result’ was considered, as well as an overall measure of performance. The data from the remaining panels were analysed, and the level of performance was recorded for each of the stated criteria. Results indicated different levels of performance, and the research also revealed the importance of choice of validation panels and the screening of samples prior to testing. A simpler performance scheme was proposed to address issues relating to attaching arbitrary weightings to each of the performance criteria, and to address potential problems associated with combining different measurement criteria into a single performance score. While there is still potential for further development and refinement, ProfiSens has made a significant contribution to proficiency testing for sensory analysis.",
keywords = "Proficiency testing, Panel performance, Ranking, Expected results",
author = "McEwan, {Jean A.} and Raija-Liisa Heini{\"o} and Hunter, {E. Anthony} and Per Lea",
year = "2003",
doi = "10.1016/S0950-3293(02)00083-6",
language = "English",
volume = "14",
pages = "247--256",
journal = "Food Quality and Preference",
issn = "0950-3293",
publisher = "Elsevier",
number = "3",

}

Proficiency testing for sensory ranking panels : Measuring panel performance. / McEwan, Jean A. (Corresponding Author); Heiniö, Raija-Liisa; Hunter, E. Anthony; Lea, Per.

In: Food Quality and Preference, Vol. 14, No. 3, 2003, p. 247-256.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Proficiency testing for sensory ranking panels

T2 - Measuring panel performance

AU - McEwan, Jean A.

AU - Heiniö, Raija-Liisa

AU - Hunter, E. Anthony

AU - Lea, Per

PY - 2003

Y1 - 2003

N2 - Proficiency testing in sensory analysis is an important step towards demonstrating that results from one sensory panel are consistent with the results of other sensory panels. The uniqueness of sensory analysis poses some specific problems for measuring the proficiency of the human instrument (panel). As part of an EU supported project, ProfiSens, 14 panels undertook ranking of sweetness on five samples of apple juice. Four panels were designated ‘validation’ panels, whose data were used firstly to establish the expected ranking results, and secondly to set the performance criteria that a trained sensory panel would be expected to achieve. Four key measures of a panel’s performance were investigated: the ability to rank the samples in the correct order; the significance level associated with differences between samples; the number of pairs of samples that a panel found to be different at a specified level of significance; and the degree of agreement between assessors within a panel. For each of these criteria an ‘expected result’ was considered, as well as an overall measure of performance. The data from the remaining panels were analysed, and the level of performance was recorded for each of the stated criteria. Results indicated different levels of performance, and the research also revealed the importance of choice of validation panels and the screening of samples prior to testing. A simpler performance scheme was proposed to address issues relating to attaching arbitrary weightings to each of the performance criteria, and to address potential problems associated with combining different measurement criteria into a single performance score. While there is still potential for further development and refinement, ProfiSens has made a significant contribution to proficiency testing for sensory analysis.

AB - Proficiency testing in sensory analysis is an important step towards demonstrating that results from one sensory panel are consistent with the results of other sensory panels. The uniqueness of sensory analysis poses some specific problems for measuring the proficiency of the human instrument (panel). As part of an EU supported project, ProfiSens, 14 panels undertook ranking of sweetness on five samples of apple juice. Four panels were designated ‘validation’ panels, whose data were used firstly to establish the expected ranking results, and secondly to set the performance criteria that a trained sensory panel would be expected to achieve. Four key measures of a panel’s performance were investigated: the ability to rank the samples in the correct order; the significance level associated with differences between samples; the number of pairs of samples that a panel found to be different at a specified level of significance; and the degree of agreement between assessors within a panel. For each of these criteria an ‘expected result’ was considered, as well as an overall measure of performance. The data from the remaining panels were analysed, and the level of performance was recorded for each of the stated criteria. Results indicated different levels of performance, and the research also revealed the importance of choice of validation panels and the screening of samples prior to testing. A simpler performance scheme was proposed to address issues relating to attaching arbitrary weightings to each of the performance criteria, and to address potential problems associated with combining different measurement criteria into a single performance score. While there is still potential for further development and refinement, ProfiSens has made a significant contribution to proficiency testing for sensory analysis.

KW - Proficiency testing

KW - Panel performance

KW - Ranking

KW - Expected results

U2 - 10.1016/S0950-3293(02)00083-6

DO - 10.1016/S0950-3293(02)00083-6

M3 - Article

VL - 14

SP - 247

EP - 256

JO - Food Quality and Preference

JF - Food Quality and Preference

SN - 0950-3293

IS - 3

ER -