CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework

Shikhar Tuli, Chia Hao Li, Ritvik Sharma, Niraj K. Jha

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Recently, automated co-design of machine learning (ML) models and accelerator architectures has attracted significant attention from both the industry and academia. However, most co-design frameworks either explore a limited search space or employ suboptimal exploration techniques for simultaneous design decision investigations of the ML model and the accelerator. Furthermore, training the ML model and simulating the accelerator performance is computationally expensive. To address these limitations, this work proposes a novel neural architecture and hardware accelerator co-design framework, called CODEBench. It comprises two new benchmarking sub-frameworks, CNNBench and AccelBench, which explore expanded design spaces of convolutional neural networks (CNNs) and CNN accelerators. CNNBench leverages an advanced search technique, Bayesian Optimization using Second-order Gradients and Heteroscedastic Surrogate Model for Neural Architecture Search, to efficiently train a neural heteroscedastic surrogate model to converge to an optimal CNN architecture by employing second-order gradients. AccelBench performs cycle-accurate simulations for diverse accelerator architectures in a vast design space. With the proposed co-design method, called Bayesian Optimization using Second-order Gradients and Heteroscedastic Surrogate Model for Co-Design of CNNs and Accelerators, our best CNN-accelerator pair achieves 1.4% higher accuracy on the CIFAR-10 dataset compared to the state-of-the-art pair while enabling 59.1% lower latency and 60.8% lower energy consumption. On the ImageNet dataset, it achieves 3.7% higher Top1 accuracy at 43.8% lower latency and 11.2% lower energy consumption. CODEBench outperforms the state-of-the-art framework, i.e., Auto-NBA, by achieving 1.5% higher accuracy and 34.7× higher throughput while enabling 11.0× lower energy-delay product and 4.0× lower chip area on CIFAR-10.

Original languageEnglish (US)
Article number51
JournalACM Transactions on Embedded Computing Systems
Volume22
Issue number3
DOIs
StatePublished - Apr 20 2023
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture

Keywords

  • Active learning
  • Neural Architecture Search
  • application-specific integrated circuits
  • hardware-software co-design
  • machine learning
  • neural network accelerators

Fingerprint

Dive into the research topics of 'CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework'. Together they form a unique fingerprint.

Cite this