Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy

Kaiyu Yang; Klint Qinami; Li Fei-Fei; Jia Deng; Olga Russakovsky

doi:10.1145/3351095.3375709

Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy

Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, Olga Russakovsky

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

201 Scopus citations

Abstract

Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including ofensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the irst steps to mitigate them constructively.

Original language	English (US)
Title of host publication	FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
Publisher	Association for Computing Machinery, Inc
Pages	547-558
Number of pages	12
ISBN (Electronic)	9781450369367
DOIs	https://doi.org/10.1145/3351095.3375709
State	Published - Jan 27 2020
Event	3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020 - Barcelona, Spain Duration: Jan 27 2020 → Jan 30 2020

Publication series

Name	FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

Conference

Conference	3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020
Country/Territory	Spain
City	Barcelona
Period	1/27/20 → 1/30/20

All Science Journal Classification (ASJC) codes

General Business, Management and Accounting
General Engineering

Keywords

Computer vision
Dataset construction
Fairness
Representative datasets

Access to Document

10.1145/3351095.3375709

Cite this

Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020). Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. In FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 547-558). (FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency). Association for Computing Machinery, Inc. https://doi.org/10.1145/3351095.3375709

Yang, Kaiyu ; Qinami, Klint ; Fei-Fei, Li et al. / Towards fairer datasets : Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, Inc, 2020. pp. 547-558 (FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency).

@inproceedings{e545256c2a9347cfa6debd4678287dd1,

title = "Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy",

abstract = "Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including ofensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the irst steps to mitigate them constructively.",

keywords = "Computer vision, Dataset construction, Fairness, Representative datasets",

author = "Kaiyu Yang and Klint Qinami and Li Fei-Fei and Jia Deng and Olga Russakovsky",

year = "2020",

month = jan,

day = "27",

doi = "10.1145/3351095.3375709",

language = "English (US)",

series = "FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency",

publisher = "Association for Computing Machinery, Inc",

pages = "547--558",

booktitle = "FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency",

note = "3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020 ; Conference date: 27-01-2020 Through 30-01-2020",

}

Yang, K, Qinami, K, Fei-Fei, L, Deng, J & Russakovsky, O 2020, Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. in FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, Inc, pp. 547-558, 3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020, Barcelona, Spain, 1/27/20. https://doi.org/10.1145/3351095.3375709

Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. / Yang, Kaiyu; Qinami, Klint; Fei-Fei, Li et al.
FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, Inc, 2020. p. 547-558 (FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Towards fairer datasets

T2 - 3rd ACM Conference on Fairness, Accountability, and Transparency, FAT* 2020

AU - Yang, Kaiyu

AU - Qinami, Klint

AU - Fei-Fei, Li

AU - Deng, Jia

AU - Russakovsky, Olga

PY - 2020/1/27

Y1 - 2020/1/27

N2 - Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including ofensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the irst steps to mitigate them constructively.

AB - Computer vision technology is being used by many but remains representative of only a few. People have reported misbehavior of computer vision models, including ofensive prediction results and lower performance for underrepresented groups. Current computer vision models are typically developed using datasets consisting of manually annotated images or videos; the data and label distributions in these datasets are critical to the models' behavior. In this paper, we examine ImageNet, a large-scale ontology of images that has spurred the development of many modern computer vision methods. We consider three key factors within the person subtree of ImageNet that may lead to problematic behavior in downstream computer vision technology: (1) the stagnant concept vocabulary of WordNet, (2) the attempt at exhaustive illustration of all categories with images, and (3) the inequality of representation in the images within concepts. We seek to illuminate the root causes of these concerns and take the irst steps to mitigate them constructively.

KW - Computer vision

KW - Dataset construction

KW - Fairness

KW - Representative datasets

UR - http://www.scopus.com/inward/record.url?scp=85079661500&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85079661500&partnerID=8YFLogxK

U2 - 10.1145/3351095.3375709

DO - 10.1145/3351095.3375709

M3 - Conference contribution

T3 - FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

SP - 547

EP - 558

BT - FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

PB - Association for Computing Machinery, Inc

Y2 - 27 January 2020 through 30 January 2020

ER -

Yang K, Qinami K, Fei-Fei L, Deng J , Russakovsky O. Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy. In FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, Inc. 2020. p. 547-558. (FAT* 2020 - Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency). doi: 10.1145/3351095.3375709

Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the ImageNet hierarchy

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this