Scalable multi-label annotation

Jia Deng, Olga Russakovsky, Jonathan Krause, Michael Bernstein, Alex Berg, Li Fei-Fei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

111 Scopus citations


We study strategies for scalable multi-label annotation, or for efficiently acquiring multiple labels from humans for a collection of items. We propose an algorithm that exploits correlation, hierarchy, and sparsity of the label distribution. A case study of labeling 200 objects using 20,000 images demonstrates the effectiveness of our approach. The algorithm results in up to 6x reduction in human computation time compared to the naïve method of querying a human annotator for the presence of every object in every image.

Original languageEnglish (US)
Title of host publicationCHI 2014
Subtitle of host publicationOne of a CHInd - Conference Proceedings, 32nd Annual ACM Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
Number of pages4
ISBN (Print)9781450324731
StatePublished - 2014
Externally publishedYes
Event32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014 - Toronto, ON, Canada
Duration: Apr 26 2014May 1 2014

Publication series

NameConference on Human Factors in Computing Systems - Proceedings


Other32nd Annual ACM Conference on Human Factors in Computing Systems, CHI 2014
CityToronto, ON

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design


  • Crowdsourcing
  • Human computation


Dive into the research topics of 'Scalable multi-label annotation'. Together they form a unique fingerprint.

Cite this