The anatomy of efficient FFT and winograd convolutions on modern CPUs

Aleksandar Zlateski, Zhen Jia, Kai Li, Fredo Durand

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

Winograd-based convolution has quickly gained traction as a preferred approach to implement convolutional neural networks (ConvNet) on various hardware platforms because it could require fewer floating point operations than FFT-based or direct convolutions. In this paper, we analyze the theoretical performances of three methods (regular FFT-, Gauss-FFT-, and Winograd-based convolutions), as well as compare their highly optimized implementations on modern multi- and many-core CPUs. With all three implementations employing the same optimizations on modern CPUs, our experimental results with modern ConvNets show that the FFT-based implementations generally outperform the Winograd-based approach, which is contrary to the popular belief. To understand the results, we use a Roofline performance model to analyze the three implementations in detail, by looking at each of their computation phases and by considering not only the number of floating point operations, but also the memory bandwidth and the cache sizes. The performance analysis explains why, and under what conditions, the FFT-based implementations outperform the Winograd-based one, on modern CPUs.

Original languageEnglish (US)
Title of host publicationICS 2019 - International Conference on Supercomputing
PublisherAssociation for Computing Machinery
Pages414-424
Number of pages11
ISBN (Electronic)9781450360791
DOIs
StatePublished - Jun 26 2019
Event33rd ACM International Conference on Supercomputing, ICS 2019, held in conjunction with the Federated Computing Research Conference, FCRC 2019 - Phoenix, United States
Duration: Jun 26 2019 → …

Publication series

NameProceedings of the International Conference on Supercomputing

Conference

Conference33rd ACM International Conference on Supercomputing, ICS 2019, held in conjunction with the Federated Computing Research Conference, FCRC 2019
Country/TerritoryUnited States
CityPhoenix
Period6/26/19 → …

All Science Journal Classification (ASJC) codes

  • General Computer Science

Fingerprint

Dive into the research topics of 'The anatomy of efficient FFT and winograd convolutions on modern CPUs'. Together they form a unique fingerprint.

Cite this