A Programmable Neural-Network Inference Accelerator Based on Scalable In-Memory Computing

Hongyang Jia, Murat Ozatay, Yinqi Tang, Hossein Valavi, Rakshit Pathak, Jinseok Lee, Naveen Verma

Research output: Chapter in Book/Report/Conference proceedingConference contribution

125 Scopus citations

Abstract

This paper presents a scalable neural-network (NN) inference accelerator in 16nm, based on an array of programmable cores employing mixed-signal In-Memory Computing (IMC), digital Near-Memory Computing (NMC), and localized buffering/control. IMC achieves high energy efficiency and throughput for matrix-vector multiplications (MVMs), which dominate NNs; but, scalability poses numerous challenges, both technologically, going to advanced nodes to maintain gains over digital architectures, and architecturally, for full execution of diverse NNs. Recent demonstrations have explored integrating IMC in programmable processors [1, 2], but have not achieved IMC efficiency and throughput for full executions. The central challenge is drastically different physical design points and associated tradeoffs incurred by IMC compared to digital engines. Namely, IMC substantially increases compute energy efficiency and HW density/parallelism, but retains the overheads of HW virtualization (state and data swapping/buffering/communication across spatial/temporal computation mappings). The demonstrated architecture is co-designed with SW-mapping algorithms (encapsulated in a custom graph compiler), to provide efficiency across a broad range of mapping strategies, to overcome these overheads.

Original languageEnglish (US)
Title of host publication2021 IEEE International Solid-State Circuits Conference, ISSCC 2021 - Digest of Technical Papers
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages236-238
Number of pages3
ISBN (Electronic)9781728195490
DOIs
StatePublished - Feb 13 2021
Event2021 IEEE International Solid-State Circuits Conference, ISSCC 2021 - San Francisco, United States
Duration: Feb 13 2021Feb 22 2021

Publication series

NameDigest of Technical Papers - IEEE International Solid-State Circuits Conference
Volume64
ISSN (Print)0193-6530

Conference

Conference2021 IEEE International Solid-State Circuits Conference, ISSCC 2021
Country/TerritoryUnited States
CitySan Francisco
Period2/13/212/22/21

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'A Programmable Neural-Network Inference Accelerator Based on Scalable In-Memory Computing'. Together they form a unique fingerprint.

Cite this