CDPAM: Contrastive learning for perceptual audio similarity

Pranay Manocha, Zeyu Jin, Richard Zhang, Adam Finkelstein

Research output: Contribution to journalConference articlepeer-review

Abstract

Many speech processing methods based on deep learning require an automatic and differentiable audio metric for the loss function. The DPAM approach of Manocha et al. [1] learns a full-reference metric trained directly on human judgments, and thus correlates well with human perception. However, it requires a large number of human annotations and does not generalize well outside the range of perturbations on which it was trained. This paper introduces CDPAM – a metric that builds on and advances DPAM. The primary improvement is to combine contrastive learning and multi-dimensional representations to build robust models from limited data. In addition, we collect human judgments on triplet comparisons to improve generalization to a broader range of audio perturbations. CDPAM correlates well with human responses across nine varied datasets. We also show that adding this metric to existing speech synthesis and enhancement methods yields significant improvement, as measured by objective and subjective tests.

Original languageEnglish (US)
Pages (from-to)196-200
Number of pages5
JournalICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2021-June
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada
Duration: Jun 6 2021Jun 11 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Keywords

  • Audio quality
  • Deep metric
  • Perceptual similarity
  • Speech enhancement
  • Speech synthesis

Fingerprint

Dive into the research topics of 'CDPAM: Contrastive learning for perceptual audio similarity'. Together they form a unique fingerprint.

Cite this