A Dataset Auditing Method for Collaboratively Trained Machine Learning Models

Yangsibo Huang, Chun Yin Huang, Xiaoxiao Li, Kai Li

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Dataset auditing for machine learning (ML) models is a method to evaluate if a given dataset is used in training a model. In a Federated Learning setting where multiple institutions collaboratively train a model with their decentralized private datasets, dataset auditing can facilitate the enforcement of regulations, which provide rules for preserving privacy, but also allow users to revoke authorizations and remove their data from collaboratively trained models. This paper first proposes a set of requirements for a practical dataset auditing method, and then present a novel dataset auditing method called Ensembled Membership Auditing (EMA ). Its key idea is to leverage previously proposed Membership Inference Attack methods and to aggregate data-wise membership scores using statistic testing to audit a dataset for a ML model. We have experimentally evaluated the proposed approach with benchmark datasets, as well as 4 X-ray datasets (CBIS-DDSM, COVIDx, Child-XRay, and CXR-NIH) and 3 dermatology datasets (DERM7pt, HAM10000, and PAD-UFES-20). Our results show that EMA meet the requirements substantially better than the previous state-of-the-art method. Our code is at:https://github.com/Hazelsuko07/EMA.

Original languageEnglish (US)
Pages (from-to)2081-2090
Number of pages10
JournalIEEE Transactions on Medical Imaging
Volume42
Issue number7
DOIs
StatePublished - Jul 1 2023

All Science Journal Classification (ASJC) codes

  • Software
  • Radiological and Ultrasound Technology
  • Electrical and Electronic Engineering
  • Computer Science Applications

Keywords

  • Privacy
  • dataset auditing
  • medical image classification

Fingerprint

Dive into the research topics of 'A Dataset Auditing Method for Collaboratively Trained Machine Learning Models'. Together they form a unique fingerprint.

Cite this