ZNNi: Maximizing the Inference Throughput of 3D Convolutional Networks on CPUs and GPUs

Aleksandar Zlateski, Kisuk Lee, Hyunjune Sebastian Seung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

Sliding window convolutional networks (ConvNets) have become a popular approach to computer vision problems such as image segmentation and object detection and localization. Here we consider the parallelization of inference, i.e., the application of a previously trained ConvNet, with emphasis on 3D images. Our goal is to maximize throughput, defined as the number of output voxels computed per unit time. We propose CPU and GPU primitives for convolutional and pooling layers, which are combined to create CPU, GPU, and CPU-GPU inference algorithms. The primitives include convolution based on highly efficient padded and pruned FFTs. Our theoretical analyses and empirical tests reveal a number of interesting findings. For example, adding host RAM can be a more efficient way of increasing throughput than adding another GPU or more CPUs. Furthermore, our CPU-GPU algorithm can achieve greater throughput than the sum of CPU-only and GPU-only throughputs.

Original languageEnglish (US)
Title of host publicationProceedings of SC 2016
Subtitle of host publicationThe International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherIEEE Computer Society
Pages854-865
Number of pages12
ISBN (Electronic)9781467388153
DOIs
StatePublished - Jul 2 2016
Event2016 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016 - Salt Lake City, United States
Duration: Nov 13 2016Nov 18 2016

Publication series

NameInternational Conference for High Performance Computing, Networking, Storage and Analysis, SC
Volume0
ISSN (Print)2167-4329
ISSN (Electronic)2167-4337

Other

Other2016 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016
Country/TerritoryUnited States
CitySalt Lake City
Period11/13/1611/18/16

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Software

Fingerprint

Dive into the research topics of 'ZNNi: Maximizing the Inference Throughput of 3D Convolutional Networks on CPUs and GPUs'. Together they form a unique fingerprint.

Cite this