MemFlow: Memory-Driven Data Scheduling with Datapath Co-Design in Accelerators for Large-Scale Inference Applications

Qi Nie, Sharad Malik

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


The increasing importance of inference algorithms, such as neural networks (NNs), principle component analysis (PCA), and singular value decomposition (SVD), etc., has led to the emergence of hardware accelerators to address power-performance tradeoffs in their implementation. Their large data sets make DRAM access the bottleneck for power and performance. Private SRAM scratch-pad memory is used to mitigate the DRAM access penalty but it is a limited resource in size and bandwidth. Thus, accelerator design is not just about computation, but also how data flow is scheduled across the memory hierarchy, including DRAM, scratch-pad SRAM, and datapath registers. Current accelerator design tools automate the generation of customized datapaths to improve performance, but have limited support for reducing DRAM/SRAM accesses during the computation. In this paper, we propose a memory-driven accelerator design methodology for large-scale inference applications, to maximize data access in the datapath and SRAM. We demonstrate its efficacy using several key kernels from large-scale inference applications.

Original languageEnglish (US)
Article number8747420
Pages (from-to)1875-1888
Number of pages14
JournalIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Issue number9
StatePublished - Sep 2020

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design
  • Electrical and Electronic Engineering


  • Accelerator
  • data scheduling
  • hardware/software co-design
  • large-scale computing
  • memory utilization


Dive into the research topics of 'MemFlow: Memory-Driven Data Scheduling with Datapath Co-Design in Accelerators for Large-Scale Inference Applications'. Together they form a unique fingerprint.

Cite this