Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification

Shuai Wang, Chengyang Li, Derrick Wing Kwan Ng, Yonina C. Eldar, H. Vincent Poor, Qi Hao, Chengzhong Xu

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


Realizing human-like perception is a challenge in open driving scenarios due to corner cases and visual occlusions. To gather knowledge of rare and occluded instances, federated learning assisted connected autonomous vehicle (FLCAV) has been proposed, which leverages vehicular networks to establish federated deep neural networks (DNNs) from distributed data captured by vehicles and road sensors. Without the need of data aggregation, FLCAV helps preserve privacy while reducing communication costs compared with conventional centralized learning. However, it is challenging to determine the network resources and road sensor placements for multi-stage training with multi-modal datasets in multi-variant scenarios. This article presents networking and training frameworks for FLCAV perception. Multi-layer graph resource allocation and vehicle-road contrastive sensor placement are proposed to address the network management and sensor deployment problems, respectively. We also develop CarlaFLCAV, a software platform that implements the above system and methods. Experimental results confirm the superiority of the proposed techniques compared with various benchmarks.

Original languageEnglish (US)
Pages (from-to)16-25
Number of pages10
JournalIEEE Network
Issue number3
StatePublished - May 1 2023

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Federated Deep Learning Meets Autonomous Vehicle Perception: Design and Verification'. Together they form a unique fingerprint.

Cite this