Crowdsourcing annotations for visual object detection

Hao Su, Jia Deng, Li Fei-Fei

Research output: Chapter in Book/Report/Conference proceedingConference contribution

80 Scopus citations

Abstract

A large number of images with ground truth object bounding boxes are critical for learning object detectors, which is a fundamental task in compute vision. In this paper, we study strategies to crowd-source bounding box annotations. The core challenge of building such a system is to effectively control the data quality with minimal cost. Our key observation is that drawing a bounding box is significantly more difficult and time consuming than giving answers to multiple choice questions. Thus quality control through additional verification tasks is more cost effective than consensus based algorithms. In particular, we present a system that consists of three simple sub-tasks - a drawing task, a quality verification task and a coverage verification task. Experimental results demonstrate that our system is scalable, accurate, and cost-effective.

Original languageEnglish (US)
Title of host publicationHuman Computation - Papers from the 2012 AAAI Workshop, Technical Report
Pages40-46
Number of pages7
StatePublished - Dec 1 2012
Externally publishedYes
Event2012 AAAI Workshop - Toronto, ON, Canada
Duration: Jul 23 2012Jul 23 2012

Publication series

NameAAAI Workshop - Technical Report
VolumeWS-12-08

Other

Other2012 AAAI Workshop
CountryCanada
CityToronto, ON
Period7/23/127/23/12

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Fingerprint Dive into the research topics of 'Crowdsourcing annotations for visual object detection'. Together they form a unique fingerprint.

  • Cite this

    Su, H., Deng, J., & Fei-Fei, L. (2012). Crowdsourcing annotations for visual object detection. In Human Computation - Papers from the 2012 AAAI Workshop, Technical Report (pp. 40-46). (AAAI Workshop - Technical Report; Vol. WS-12-08).