Building Efficient Neural Prefetcher

Yuchen Liu, Georgios Tziantzioulis, David Wentzlaff

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Data prefetching is a promising approach to mitigate computation slow down due to the memory wall. While modern workloads grow more and more complicated, their memory access patterns become less organized and rule-based prefetchers can no longer deliver improved performance, which motivates the research of adopting neural networks for prefetching. However, current neural prefetchers require high computation costs and large storage space to obtain good performance, which makes them far from practical. To this end, we address the efficiency issue in neural prefetchers and propose an effective approach to build light-weight models. Specifically, our method is aware of both machine learning and micro-architecture, where we introduce a novel neural prefetcher design space with knobs from both aspects. We optimize these knobs using workload characteristic observations, rigorous mathematical optimization, and efficient design space traversal, which provides us with highly-efficient neural prefetchers. Our approach is evaluated on SPEC CPU 2006, where our models can provide up to 60% IPC gain compared to no prefetching, outperforming non-neural based prefetchers. In comparison with the state of the art neural prefetcher, our models enjoy an average of 15.4 × multiply-accumulation reduction, 6.7 × parameters saving, with even better IPC gains. Although it is still challenging to provide implementable neural prefetcher, this order of magnitude computational and storage reduction, provided by our method, marks an important milestone towards practical neural prefetchers.

Original languageEnglish (US)
Title of host publicationMEMSYS 2023 - Proceedings of the International Symposium on Memory Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9798400716447
StatePublished - Oct 2 2023
Event9th International Symposium on Memory Systems, MEMSYS 2023 - Washington, United States
Duration: Oct 2 2023Oct 5 2023

Publication series

NameACM International Conference Proceeding Series


Conference9th International Symposium on Memory Systems, MEMSYS 2023
Country/TerritoryUnited States

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software


Dive into the research topics of 'Building Efficient Neural Prefetcher'. Together they form a unique fingerprint.

Cite this