A Unified Framework of Surrogate Loss by Refactoring and Interpolation

Lanlan Liu, Mingzhe Wang, Jia Deng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We introduce UniLoss, a unified framework to generate surrogate losses for training deep networks with gradient descent, reducing the amount of manual design of task-specific surrogate losses. Our key observation is that in many cases, evaluating a model with a performance metric on a batch of examples can be refactored into four steps: from input to real-valued scores, from scores to comparisons of pairs of scores, from comparisons to binary variables, and from binary variables to the final performance metric. Using this refactoring we generate differentiable approximations for each non-differentiable step through interpolation. Using UniLoss, we can optimize for different tasks and metrics using one unified framework, achieving comparable performance compared with task-specific losses. We validate the effectiveness of UniLoss on three tasks and four datasets. Code is available at https://github.com/princeton-vl/uniloss.

Original languageEnglish (US)
Title of host publicationComputer Vision – ECCV 2020 - 16th European Conference 2020, Proceedings
EditorsAndrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
PublisherSpringer Science and Business Media Deutschland GmbH
Pages278-293
Number of pages16
ISBN (Print)9783030585792
DOIs
StatePublished - 2020
Event16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
Duration: Aug 23 2020Aug 28 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12348 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference16th European Conference on Computer Vision, ECCV 2020
CountryUnited Kingdom
CityGlasgow
Period8/23/208/28/20

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Keywords

  • Image classification
  • Loss design
  • Pose estimation

Fingerprint Dive into the research topics of 'A Unified Framework of Surrogate Loss by Refactoring and Interpolation'. Together they form a unique fingerprint.

Cite this