Interpretable and Learnable Super-Resolution Time-Frequency Representation

Randall Balestriero, Hervé Glotin, Richard G. Baraniuk

Research output: Contribution to journalConference articlepeer-review


We develop a novel interpretable and learnable time-frequency representation (TFR) that produces a super-resolved quadratic signal representation for time-series analysis; the proposed TFR is a Gaussian filtering of the Wigner-Ville (WV) transform of a signal parametrized with a few interpretable parameters. Our approach has two main hallmarks. First, by varying the filters applied onto the WV, our new TFR can interpolate between known TFRs such as spectrograms, wavelet transforms, and chirplet transforms. Beyond that, our representation can also reach perfect time and frequency localization, hence super-resolution; this generalizes standard TFRs whose resolution is limited by the Heisenberg uncertainty principle. Second, our proposed TFR is interpretable thanks to an explicit low-dimensional and physical parametrization of the WV Gaussian filtering. We demonstrate that our approach enables us to learn highly adapted TFRs and is able to tackle a range of large-scale classification tasks, where we reach higher performance compared to baseline and learned TFRs. Ours is to the best of our knowledge the first learnable TFR that can continuously interpolate between super-resolution representation and commonly employed TFRs based on a few learnable parameters and which preserves full interpretability of the produced TFR, even after learning.

Original languageEnglish (US)
Pages (from-to)118-152
Number of pages35
JournalProceedings of Machine Learning Research
StatePublished - 2021
Event2nd Mathematical and Scientific Machine Learning Conference, MSML 2021 - Virtual, Online
Duration: Aug 16 2021Aug 19 2021


  • Audio Classification
  • bioacoustics
  • biosonar
  • Chirplet
  • Cohen Class
  • Environmental Sounds
  • Explain-ability
  • Gabor Transform
  • Interpretability
  • Learnable Time-Frequency Representation
  • Spectrogram
  • Speech Recognition
  • Super-Resolution
  • Time-Series
  • Wavelet
  • Wigner-Ville

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability


Dive into the research topics of 'Interpretable and Learnable Super-Resolution Time-Frequency Representation'. Together they form a unique fingerprint.

Cite this