On Wiener filtering and the physics behind statistical modeling

Ralf Marbach

Research output: Contribution to journalArticleScientificpeer-review

34 Citations (Scopus)

Abstract

The closed-form solution of the so-called statistical multivariate calibration model is given in terms of the pure component spectral signal, the spectral noise, and the signal and noise of the reference method.
The ‘‘statistical’’ calibration model is shown to be as much grounded on the physics of the pure component spectra as any of the ‘‘physical’’ models. There are no fundamental differences between the two approaches since both are merely different attempts to realize the same basic idea, viz., the spectrometric Wiener filter.
The concept of the application-specific signal-to-noise ratio (SNR) is introduced, which is a combination of the two SNRs from the reference and the spectral data. Both are defined and the central importance of the latter for the assessment and development of spectroscopic instruments and methods is explained.
Other statistics like the correlation coefficient, prediction error, slope deficiency, etc., are functions of the SNR. Spurious correlations and other practically important issues are discussed in quantitative terms. Most important, it is shown how to use a priori information about the pure component spectra and the spectral noise in an optimal way, thereby making the distinction between statistical and physical calibrations obsolete and combining the best of both worlds.
Companies and research groups can use this article to realize significant savings in cost and time for development efforts.
Original languageEnglish
Pages (from-to)130-147
JournalJournal of Biomedical Optics
Volume7
Issue number1
DOIs
Publication statusPublished - 2002
MoE publication typeA1 Journal article-refereed

Fingerprint

Wiener filtering
Physics
Calibration
white noise
physics
Signal to noise ratio
signal to noise ratios
correlation coefficients
Statistics
statistics
slopes
costs
filters
predictions
Costs
Industry

Keywords

  • Wiener filters
  • calibration
  • optical noise
  • filtering theory
  • bio-optics
  • statistical analysis
  • light absorption

Cite this

@article{957aaa504d0146a4877de0dd46811697,
title = "On Wiener filtering and the physics behind statistical modeling",
abstract = "The closed-form solution of the so-called statistical multivariate calibration model is given in terms of the pure component spectral signal, the spectral noise, and the signal and noise of the reference method. The ‘‘statistical’’ calibration model is shown to be as much grounded on the physics of the pure component spectra as any of the ‘‘physical’’ models. There are no fundamental differences between the two approaches since both are merely different attempts to realize the same basic idea, viz., the spectrometric Wiener filter. The concept of the application-specific signal-to-noise ratio (SNR) is introduced, which is a combination of the two SNRs from the reference and the spectral data. Both are defined and the central importance of the latter for the assessment and development of spectroscopic instruments and methods is explained. Other statistics like the correlation coefficient, prediction error, slope deficiency, etc., are functions of the SNR. Spurious correlations and other practically important issues are discussed in quantitative terms. Most important, it is shown how to use a priori information about the pure component spectra and the spectral noise in an optimal way, thereby making the distinction between statistical and physical calibrations obsolete and combining the best of both worlds. Companies and research groups can use this article to realize significant savings in cost and time for development efforts.",
keywords = "Wiener filters, calibration, optical noise, filtering theory, bio-optics, statistical analysis, light absorption",
author = "Ralf Marbach",
year = "2002",
doi = "10.1117/1.1427051",
language = "English",
volume = "7",
pages = "130--147",
journal = "Journal of Biomedical Optics",
issn = "1083-3668",
publisher = "International Society for Optics and Photonics SPIE",
number = "1",

}

On Wiener filtering and the physics behind statistical modeling. / Marbach, Ralf.

In: Journal of Biomedical Optics, Vol. 7, No. 1, 2002, p. 130-147.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - On Wiener filtering and the physics behind statistical modeling

AU - Marbach, Ralf

PY - 2002

Y1 - 2002

N2 - The closed-form solution of the so-called statistical multivariate calibration model is given in terms of the pure component spectral signal, the spectral noise, and the signal and noise of the reference method. The ‘‘statistical’’ calibration model is shown to be as much grounded on the physics of the pure component spectra as any of the ‘‘physical’’ models. There are no fundamental differences between the two approaches since both are merely different attempts to realize the same basic idea, viz., the spectrometric Wiener filter. The concept of the application-specific signal-to-noise ratio (SNR) is introduced, which is a combination of the two SNRs from the reference and the spectral data. Both are defined and the central importance of the latter for the assessment and development of spectroscopic instruments and methods is explained. Other statistics like the correlation coefficient, prediction error, slope deficiency, etc., are functions of the SNR. Spurious correlations and other practically important issues are discussed in quantitative terms. Most important, it is shown how to use a priori information about the pure component spectra and the spectral noise in an optimal way, thereby making the distinction between statistical and physical calibrations obsolete and combining the best of both worlds. Companies and research groups can use this article to realize significant savings in cost and time for development efforts.

AB - The closed-form solution of the so-called statistical multivariate calibration model is given in terms of the pure component spectral signal, the spectral noise, and the signal and noise of the reference method. The ‘‘statistical’’ calibration model is shown to be as much grounded on the physics of the pure component spectra as any of the ‘‘physical’’ models. There are no fundamental differences between the two approaches since both are merely different attempts to realize the same basic idea, viz., the spectrometric Wiener filter. The concept of the application-specific signal-to-noise ratio (SNR) is introduced, which is a combination of the two SNRs from the reference and the spectral data. Both are defined and the central importance of the latter for the assessment and development of spectroscopic instruments and methods is explained. Other statistics like the correlation coefficient, prediction error, slope deficiency, etc., are functions of the SNR. Spurious correlations and other practically important issues are discussed in quantitative terms. Most important, it is shown how to use a priori information about the pure component spectra and the spectral noise in an optimal way, thereby making the distinction between statistical and physical calibrations obsolete and combining the best of both worlds. Companies and research groups can use this article to realize significant savings in cost and time for development efforts.

KW - Wiener filters

KW - calibration

KW - optical noise

KW - filtering theory

KW - bio-optics

KW - statistical analysis

KW - light absorption

U2 - 10.1117/1.1427051

DO - 10.1117/1.1427051

M3 - Article

VL - 7

SP - 130

EP - 147

JO - Journal of Biomedical Optics

JF - Journal of Biomedical Optics

SN - 1083-3668

IS - 1

ER -