The fourth phase of the radiative transfer model intercomparison (RAMI) exercise

Actual canopy scenarios and conformity testing

Jean Luc Widlowski, Corrado Mio, Mathias Disney, Jennifer Adams, Ioannis Andredakis, Clement Atzberger, James Brennan, Lorenzo Busetto, Michaël Chelle, Guido Ceccherini, Roberto Colombo, Jean Francois Côté, Alo Eenmäe, Richard Essery, Jean Philippe Gastellu-Etchegorry, Nadine Gobron, Eloi Grau, Vanessa Haverd, Lucie Homolová, Huaguo Huang & 22 others Linda Hunt, Hideki Kobayashi, Benjamin Koetz, Andres Kuusk, Joel Kuusk, Mait Lang, Philip E. Lewis, Jennifer L. Lovell, Zbyněk Malenovský, Michele Meroni, Felix Morsdorf, Matti Mõttus, Wenge Ni-Meister, Bernard Pinty, Miina Rautiainen, Martin Schlerf, Ben Somers, Jan Stuckens, Michel M. Verstraete, Wenze Yang, Feng Zhao, Terenzio Zenone

Research output: Contribution to journalArticleScientificpeer-review

65 Citations (Scopus)

Abstract

The RAdiative transfer Model Intercomparison (RAMI) activity focuses on the benchmarking of canopy radiative transfer (RT) models. For the current fourth phase of RAMI, six highly realistic virtual plant environments were constructed on the basis of intensive field data collected from (both deciduous and coniferous) forest stands as well as test sites in Europe and South Africa. Twelve RT modelling groups provided simulations of canopy scale (directional and hemispherically integrated) radiative quantities, as well as a series of binary hemispherical photographs acquired from different locations within the virtual canopies. The simulation results showed much greater variance than those recently analysed for the abstract canopy scenarios of RAMI-IV. Canopy complexity is among the most likely drivers behind operator induced errors that gave rise to the discrepancies. Conformity testing was introduced to separate the simulation results into acceptable and non-acceptable contributions. More specifically, a shared risk approach is used to evaluate the compliance of RT model simulations on the basis of reference data generated with the weighted ensemble averaging technique from ISO-13528. However, using concepts from legal metrology, the uncertainty of this reference solution will be shown to prevent a confident assessment of model performance with respect to the selected tolerance intervals. As an alternative, guarded risk decision rules will be presented to account explicitly for the uncertainty associated with the reference and candidate methods. Both guarded acceptance and guarded rejection approaches are used to make confident statements about the acceptance and/or rejection of RT model simulations with respect to the predefined tolerance intervals.

Original languageEnglish
Pages (from-to)418-437
Number of pages20
JournalRemote Sensing of Environment
Volume169
DOIs
Publication statusPublished - 1 Nov 2015
MoE publication typeA1 Journal article-refereed

Fingerprint

Radiative transfer
radiative transfer
exercise
canopy
Testing
testing
simulation
simulation models
uncertainty
tolerance
metrology
benchmarking
Benchmarking
coniferous forest
deciduous forests
forest stands
compliance
photographs
coniferous forests
photograph

Keywords

  • 3D virtual plant canopy
  • Conformity testing
  • Digital hemispherical photography
  • GCOS
  • Guarded acceptance
  • ISO-13528
  • Model benchmarking
  • Optical remote sensing
  • Radiative transfer
  • Shared risk

Cite this

Widlowski, Jean Luc ; Mio, Corrado ; Disney, Mathias ; Adams, Jennifer ; Andredakis, Ioannis ; Atzberger, Clement ; Brennan, James ; Busetto, Lorenzo ; Chelle, Michaël ; Ceccherini, Guido ; Colombo, Roberto ; Côté, Jean Francois ; Eenmäe, Alo ; Essery, Richard ; Gastellu-Etchegorry, Jean Philippe ; Gobron, Nadine ; Grau, Eloi ; Haverd, Vanessa ; Homolová, Lucie ; Huang, Huaguo ; Hunt, Linda ; Kobayashi, Hideki ; Koetz, Benjamin ; Kuusk, Andres ; Kuusk, Joel ; Lang, Mait ; Lewis, Philip E. ; Lovell, Jennifer L. ; Malenovský, Zbyněk ; Meroni, Michele ; Morsdorf, Felix ; Mõttus, Matti ; Ni-Meister, Wenge ; Pinty, Bernard ; Rautiainen, Miina ; Schlerf, Martin ; Somers, Ben ; Stuckens, Jan ; Verstraete, Michel M. ; Yang, Wenze ; Zhao, Feng ; Zenone, Terenzio. / The fourth phase of the radiative transfer model intercomparison (RAMI) exercise : Actual canopy scenarios and conformity testing. In: Remote Sensing of Environment. 2015 ; Vol. 169. pp. 418-437.
@article{9e9cb62dbce14f959ab3baab7c3b53cf,
title = "The fourth phase of the radiative transfer model intercomparison (RAMI) exercise: Actual canopy scenarios and conformity testing",
abstract = "The RAdiative transfer Model Intercomparison (RAMI) activity focuses on the benchmarking of canopy radiative transfer (RT) models. For the current fourth phase of RAMI, six highly realistic virtual plant environments were constructed on the basis of intensive field data collected from (both deciduous and coniferous) forest stands as well as test sites in Europe and South Africa. Twelve RT modelling groups provided simulations of canopy scale (directional and hemispherically integrated) radiative quantities, as well as a series of binary hemispherical photographs acquired from different locations within the virtual canopies. The simulation results showed much greater variance than those recently analysed for the abstract canopy scenarios of RAMI-IV. Canopy complexity is among the most likely drivers behind operator induced errors that gave rise to the discrepancies. Conformity testing was introduced to separate the simulation results into acceptable and non-acceptable contributions. More specifically, a shared risk approach is used to evaluate the compliance of RT model simulations on the basis of reference data generated with the weighted ensemble averaging technique from ISO-13528. However, using concepts from legal metrology, the uncertainty of this reference solution will be shown to prevent a confident assessment of model performance with respect to the selected tolerance intervals. As an alternative, guarded risk decision rules will be presented to account explicitly for the uncertainty associated with the reference and candidate methods. Both guarded acceptance and guarded rejection approaches are used to make confident statements about the acceptance and/or rejection of RT model simulations with respect to the predefined tolerance intervals.",
keywords = "3D virtual plant canopy, Conformity testing, Digital hemispherical photography, GCOS, Guarded acceptance, ISO-13528, Model benchmarking, Optical remote sensing, Radiative transfer, Shared risk",
author = "Widlowski, {Jean Luc} and Corrado Mio and Mathias Disney and Jennifer Adams and Ioannis Andredakis and Clement Atzberger and James Brennan and Lorenzo Busetto and Micha{\"e}l Chelle and Guido Ceccherini and Roberto Colombo and C{\^o}t{\'e}, {Jean Francois} and Alo Eenm{\"a}e and Richard Essery and Gastellu-Etchegorry, {Jean Philippe} and Nadine Gobron and Eloi Grau and Vanessa Haverd and Lucie Homolov{\'a} and Huaguo Huang and Linda Hunt and Hideki Kobayashi and Benjamin Koetz and Andres Kuusk and Joel Kuusk and Mait Lang and Lewis, {Philip E.} and Lovell, {Jennifer L.} and Zbyněk Malenovsk{\'y} and Michele Meroni and Felix Morsdorf and Matti M{\~o}ttus and Wenge Ni-Meister and Bernard Pinty and Miina Rautiainen and Martin Schlerf and Ben Somers and Jan Stuckens and Verstraete, {Michel M.} and Wenze Yang and Feng Zhao and Terenzio Zenone",
year = "2015",
month = "11",
day = "1",
doi = "10.1016/j.rse.2015.08.016",
language = "English",
volume = "169",
pages = "418--437",
journal = "Remote Sensing of Environment",
issn = "0034-4257",
publisher = "Elsevier",

}

Widlowski, JL, Mio, C, Disney, M, Adams, J, Andredakis, I, Atzberger, C, Brennan, J, Busetto, L, Chelle, M, Ceccherini, G, Colombo, R, Côté, JF, Eenmäe, A, Essery, R, Gastellu-Etchegorry, JP, Gobron, N, Grau, E, Haverd, V, Homolová, L, Huang, H, Hunt, L, Kobayashi, H, Koetz, B, Kuusk, A, Kuusk, J, Lang, M, Lewis, PE, Lovell, JL, Malenovský, Z, Meroni, M, Morsdorf, F, Mõttus, M, Ni-Meister, W, Pinty, B, Rautiainen, M, Schlerf, M, Somers, B, Stuckens, J, Verstraete, MM, Yang, W, Zhao, F & Zenone, T 2015, 'The fourth phase of the radiative transfer model intercomparison (RAMI) exercise: Actual canopy scenarios and conformity testing', Remote Sensing of Environment, vol. 169, pp. 418-437. https://doi.org/10.1016/j.rse.2015.08.016

The fourth phase of the radiative transfer model intercomparison (RAMI) exercise : Actual canopy scenarios and conformity testing. / Widlowski, Jean Luc; Mio, Corrado; Disney, Mathias; Adams, Jennifer; Andredakis, Ioannis; Atzberger, Clement; Brennan, James; Busetto, Lorenzo; Chelle, Michaël; Ceccherini, Guido; Colombo, Roberto; Côté, Jean Francois; Eenmäe, Alo; Essery, Richard; Gastellu-Etchegorry, Jean Philippe; Gobron, Nadine; Grau, Eloi; Haverd, Vanessa; Homolová, Lucie; Huang, Huaguo; Hunt, Linda; Kobayashi, Hideki; Koetz, Benjamin; Kuusk, Andres; Kuusk, Joel; Lang, Mait; Lewis, Philip E.; Lovell, Jennifer L.; Malenovský, Zbyněk; Meroni, Michele; Morsdorf, Felix; Mõttus, Matti; Ni-Meister, Wenge; Pinty, Bernard; Rautiainen, Miina; Schlerf, Martin; Somers, Ben; Stuckens, Jan; Verstraete, Michel M.; Yang, Wenze; Zhao, Feng; Zenone, Terenzio.

In: Remote Sensing of Environment, Vol. 169, 01.11.2015, p. 418-437.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - The fourth phase of the radiative transfer model intercomparison (RAMI) exercise

T2 - Actual canopy scenarios and conformity testing

AU - Widlowski, Jean Luc

AU - Mio, Corrado

AU - Disney, Mathias

AU - Adams, Jennifer

AU - Andredakis, Ioannis

AU - Atzberger, Clement

AU - Brennan, James

AU - Busetto, Lorenzo

AU - Chelle, Michaël

AU - Ceccherini, Guido

AU - Colombo, Roberto

AU - Côté, Jean Francois

AU - Eenmäe, Alo

AU - Essery, Richard

AU - Gastellu-Etchegorry, Jean Philippe

AU - Gobron, Nadine

AU - Grau, Eloi

AU - Haverd, Vanessa

AU - Homolová, Lucie

AU - Huang, Huaguo

AU - Hunt, Linda

AU - Kobayashi, Hideki

AU - Koetz, Benjamin

AU - Kuusk, Andres

AU - Kuusk, Joel

AU - Lang, Mait

AU - Lewis, Philip E.

AU - Lovell, Jennifer L.

AU - Malenovský, Zbyněk

AU - Meroni, Michele

AU - Morsdorf, Felix

AU - Mõttus, Matti

AU - Ni-Meister, Wenge

AU - Pinty, Bernard

AU - Rautiainen, Miina

AU - Schlerf, Martin

AU - Somers, Ben

AU - Stuckens, Jan

AU - Verstraete, Michel M.

AU - Yang, Wenze

AU - Zhao, Feng

AU - Zenone, Terenzio

PY - 2015/11/1

Y1 - 2015/11/1

N2 - The RAdiative transfer Model Intercomparison (RAMI) activity focuses on the benchmarking of canopy radiative transfer (RT) models. For the current fourth phase of RAMI, six highly realistic virtual plant environments were constructed on the basis of intensive field data collected from (both deciduous and coniferous) forest stands as well as test sites in Europe and South Africa. Twelve RT modelling groups provided simulations of canopy scale (directional and hemispherically integrated) radiative quantities, as well as a series of binary hemispherical photographs acquired from different locations within the virtual canopies. The simulation results showed much greater variance than those recently analysed for the abstract canopy scenarios of RAMI-IV. Canopy complexity is among the most likely drivers behind operator induced errors that gave rise to the discrepancies. Conformity testing was introduced to separate the simulation results into acceptable and non-acceptable contributions. More specifically, a shared risk approach is used to evaluate the compliance of RT model simulations on the basis of reference data generated with the weighted ensemble averaging technique from ISO-13528. However, using concepts from legal metrology, the uncertainty of this reference solution will be shown to prevent a confident assessment of model performance with respect to the selected tolerance intervals. As an alternative, guarded risk decision rules will be presented to account explicitly for the uncertainty associated with the reference and candidate methods. Both guarded acceptance and guarded rejection approaches are used to make confident statements about the acceptance and/or rejection of RT model simulations with respect to the predefined tolerance intervals.

AB - The RAdiative transfer Model Intercomparison (RAMI) activity focuses on the benchmarking of canopy radiative transfer (RT) models. For the current fourth phase of RAMI, six highly realistic virtual plant environments were constructed on the basis of intensive field data collected from (both deciduous and coniferous) forest stands as well as test sites in Europe and South Africa. Twelve RT modelling groups provided simulations of canopy scale (directional and hemispherically integrated) radiative quantities, as well as a series of binary hemispherical photographs acquired from different locations within the virtual canopies. The simulation results showed much greater variance than those recently analysed for the abstract canopy scenarios of RAMI-IV. Canopy complexity is among the most likely drivers behind operator induced errors that gave rise to the discrepancies. Conformity testing was introduced to separate the simulation results into acceptable and non-acceptable contributions. More specifically, a shared risk approach is used to evaluate the compliance of RT model simulations on the basis of reference data generated with the weighted ensemble averaging technique from ISO-13528. However, using concepts from legal metrology, the uncertainty of this reference solution will be shown to prevent a confident assessment of model performance with respect to the selected tolerance intervals. As an alternative, guarded risk decision rules will be presented to account explicitly for the uncertainty associated with the reference and candidate methods. Both guarded acceptance and guarded rejection approaches are used to make confident statements about the acceptance and/or rejection of RT model simulations with respect to the predefined tolerance intervals.

KW - 3D virtual plant canopy

KW - Conformity testing

KW - Digital hemispherical photography

KW - GCOS

KW - Guarded acceptance

KW - ISO-13528

KW - Model benchmarking

KW - Optical remote sensing

KW - Radiative transfer

KW - Shared risk

UR - http://www.scopus.com/inward/record.url?scp=84928727193&partnerID=8YFLogxK

U2 - 10.1016/j.rse.2015.08.016

DO - 10.1016/j.rse.2015.08.016

M3 - Article

VL - 169

SP - 418

EP - 437

JO - Remote Sensing of Environment

JF - Remote Sensing of Environment

SN - 0034-4257

ER -