Scalable multi-label annotation

Jia Deng, Olga Russakovsky, Jonathan Krause, Michael Bernstein, Alex Berg, Li Fei-Fei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

54 Scopus citations

Abstract

We study strategies for scalable multi-label annotation, or for efficiently acquiring multiple labels from humans for a collection of items. We propose an algorithm that exploits correlation, hierarchy, and sparsity of the label distribution. A case study of labeling 200 objects using 20,000 images demonstrates the effectiveness of our approach. The algorithm results in up to 6x reduction in human computation time compared to the naïve method of querying a human annotator for the presence of every object in every image.

Original languageEnglish (US)
Title of host publicationCHI 2014
Subtitle of host publicationOne of a CHInd - Conference Proceedings, 32nd Annual ACM Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Pages3099-3102
Number of pages4
ISBN (Print)9781450324731
DOIs
StatePublished - Jan 1 2014
Externally publishedYes
Event32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014 - Toronto, ON, Canada
Duration: Apr 26 2014May 1 2014

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Other

Other32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014
CountryCanada
CityToronto, ON
Period4/26/145/1/14

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Keywords

  • Crowdsourcing
  • Human computation

Fingerprint Dive into the research topics of 'Scalable multi-label annotation'. Together they form a unique fingerprint.

  • Cite this

    Deng, J., Russakovsky, O., Krause, J., Bernstein, M., Berg, A., & Fei-Fei, L. (2014). Scalable multi-label annotation. In CHI 2014: One of a CHInd - Conference Proceedings, 32nd Annual ACM Conference on Human Factors in Computing Systems (pp. 3099-3102). (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/2556288.2557011