Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

Mikko Vehkapera, Yoshiyuki Kabashima, Saikat Chatterjee

Research output: Contribution to journalArticleScientificpeer-review

15 Citations (Scopus)

Abstract

The performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric, and so-called $T$-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on $\ell-{1}$-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to $\ell-{2}$-norm and zero-norm regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns, the $T$-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

Original languageEnglish
Article number7399380
Pages (from-to)2100-2124
Number of pages25
JournalIEEE Transactions on Information Theory
Volume62
Issue number4
DOIs
Publication statusPublished - 1 Apr 2016
MoE publication typeA1 Journal article-refereed

Fingerprint

Compressed sensing
reconstruction
experiment
system model
performance
scenario
Additive noise
Mean square error
Experiments

Keywords

  • '1 minimization
  • Compressed sensing
  • compressed sensing matrices
  • eigenvalues of random matrices
  • noisy linear measurements

Cite this

@article{70e2466cd8f64c48ad64485aaaecae25,
title = "Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing",
abstract = "The performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric, and so-called $T$-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on $\ell-{1}$-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to $\ell-{2}$-norm and zero-norm regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns, the $T$-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.",
keywords = "'1 minimization, Compressed sensing, compressed sensing matrices, eigenvalues of random matrices, noisy linear measurements",
author = "Mikko Vehkapera and Yoshiyuki Kabashima and Saikat Chatterjee",
year = "2016",
month = "4",
day = "1",
doi = "10.1109/TIT.2016.2525824",
language = "English",
volume = "62",
pages = "2100--2124",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "IEEE Institute of Electrical and Electronic Engineers",
number = "4",

}

Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing. / Vehkapera, Mikko; Kabashima, Yoshiyuki; Chatterjee, Saikat.

In: IEEE Transactions on Information Theory, Vol. 62, No. 4, 7399380, 01.04.2016, p. 2100-2124.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Analysis of Regularized LS Reconstruction and Random Matrix Ensembles in Compressed Sensing

AU - Vehkapera, Mikko

AU - Kabashima, Yoshiyuki

AU - Chatterjee, Saikat

PY - 2016/4/1

Y1 - 2016/4/1

N2 - The performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric, and so-called $T$-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on $\ell-{1}$-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to $\ell-{2}$-norm and zero-norm regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns, the $T$-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

AB - The performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, row-orthogonal, geometric, and so-called $T$-orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on $\ell-{1}$-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to $\ell-{2}$-norm and zero-norm regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results. The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns, the $T$-orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

KW - '1 minimization

KW - Compressed sensing

KW - compressed sensing matrices

KW - eigenvalues of random matrices

KW - noisy linear measurements

UR - http://www.scopus.com/inward/record.url?scp=84963759120&partnerID=8YFLogxK

U2 - 10.1109/TIT.2016.2525824

DO - 10.1109/TIT.2016.2525824

M3 - Article

AN - SCOPUS:84963759120

VL - 62

SP - 2100

EP - 2124

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 4

M1 - 7399380

ER -