Sketched covariance testing: A compression-statistics tradeoff

Gautam Dasarathy; Parikshit Shah; Richard G. Baraniuk

doi:10.1109/ACSSC.2017.8335428

Sketched covariance testing: A compression-statistics tradeoff

Gautam Dasarathy, Parikshit Shah, Richard G. Baraniuk

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ₀, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

Original language	English (US)
Title of host publication	Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
Editors	Michael B. Matthews
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	676-680
Number of pages	5
Volume	2017-October
ISBN (Electronic)	9781538618233
DOIs	https://doi.org/10.1109/ACSSC.2017.8335428
State	Published - Apr 10 2018
Event	51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 - Pacific Grove, United States Duration: Oct 29 2017 → Nov 1 2017

Publication series

Name	Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
Volume	2017-October

Conference

Conference	51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017
Country/Territory	United States
City	Pacific Grove
Period	10/29/17 → 11/1/17

ASJC Scopus subject areas

Control and Optimization
Computer Networks and Communications
Hardware and Architecture
Signal Processing
Biomedical Engineering
Instrumentation

Access to Document

10.1109/ACSSC.2017.8335428

Cite this

Dasarathy, G., Shah, P., & Baraniuk, R. G. (2018). Sketched covariance testing: A compression-statistics tradeoff. In M. B. Matthews (Ed.), Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 (Vol. 2017-October, pp. 676-680). (Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACSSC.2017.8335428

Sketched covariance testing: A compression-statistics tradeoff. / Dasarathy, Gautam; Shah, Parikshit; Baraniuk, Richard G.
Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. ed. / Michael B. Matthews. Vol. 2017-October Institute of Electrical and Electronics Engineers Inc., 2018. p. 676-680 (Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017; Vol. 2017-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Dasarathy, G, Shah, P & Baraniuk, RG 2018, Sketched covariance testing: A compression-statistics tradeoff. in MB Matthews (ed.), Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. vol. 2017-October, Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017, vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 676-680, 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017, Pacific Grove, United States, 10/29/17. https://doi.org/10.1109/ACSSC.2017.8335428

Dasarathy G, Shah P, Baraniuk RG. Sketched covariance testing: A compression-statistics tradeoff. In Matthews MB, editor, Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. Vol. 2017-October. Institute of Electrical and Electronics Engineers Inc. 2018. p. 676-680. (Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017). doi: 10.1109/ACSSC.2017.8335428

Dasarathy, Gautam ; Shah, Parikshit ; Baraniuk, Richard G. / Sketched covariance testing : A compression-statistics tradeoff. Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017. editor / Michael B. Matthews. Vol. 2017-October Institute of Electrical and Electronics Engineers Inc., 2018. pp. 676-680 (Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017).

@inproceedings{740b5bc460ae43048753ca7fec1e21a9,

title = "Sketched covariance testing: A compression-statistics tradeoff",

abstract = "Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.",

author = "Gautam Dasarathy and Parikshit Shah and Baraniuk, {Richard G.}",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE. Copyright: Copyright 2018 Elsevier B.V., All rights reserved.; 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017 ; Conference date: 29-10-2017 Through 01-11-2017",

year = "2018",

month = apr,

day = "10",

doi = "10.1109/ACSSC.2017.8335428",

language = "English (US)",

volume = "2017-October",

series = "Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "676--680",

editor = "Matthews, {Michael B.}",

booktitle = "Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017",

address = "United States",

}

TY - GEN

T1 - Sketched covariance testing

T2 - 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017

AU - Dasarathy, Gautam

AU - Shah, Parikshit

AU - Baraniuk, Richard G.

PY - 2018/4/10

Y1 - 2018/4/10

N2 - Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

AB - Hypothesis testing of covariance matrices is an important problem in multivariate analysis. Given n data samples and a covariance matrix Σ0, the goal is to determine whether or not the data is consistent with this matrix. In this paper we introduce a framework that we call sketched covariance testing, where the data is provided after being compressed by multiplying by a 'sketching' matrix A chosen by the analyst. We propose a statistical test in this setting and quantify an achievable sample complexity as a function of the amount of compression. Our result reveals an intriguing achievable tradeoff between the compression ratio and the statistical information required for reliable hypothesis testing; the sample complexity increases as the fourth power of the amount of compression.

UR - http://www.scopus.com/inward/record.url?scp=85050943633&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050943633&partnerID=8YFLogxK

U2 - 10.1109/ACSSC.2017.8335428

DO - 10.1109/ACSSC.2017.8335428

M3 - Conference contribution

AN - SCOPUS:85050943633

VL - 2017-October

T3 - Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017

SP - 676

EP - 680

BT - Conference Record of 51st Asilomar Conference on Signals, Systems and Computers, ACSSC 2017

A2 - Matthews, Michael B.

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 29 October 2017 through 1 November 2017

ER -

Sketched covariance testing: A compression-statistics tradeoff

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this