MITTS: Memory Inter-arrival Time Traffic Shaping

Yanqi Zhou, David Wentzlaff

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations

Abstract

Memory bandwidth severely limits the scalability and performance of multicore and manycore systems. Application performance can be very sensitive to both the delivered memory bandwidth and latency. In multicore systems, a memory channel is usually shared by multiple cores. Having the ability to precisely provision, schedule, and isolate memory bandwidth and latency on a per-core basis is particularly important when different memory guarantees are needed on a per-customer, per-application, or per-core basis. Infrastructure as a Service (IaaS) Cloud systems, and even general purpose multicores optimized for application throughput or fairness all benefit from the ability to control and schedule memory access on a fine-grain basis. In this paper, we propose MITTS (Memory Inter-arrival Time Traffic Shaping), a simple, distributed hardware mechanism which limits memory traffic at the source (Core or LLC). MITTS shapes memory traffic based on memory request inter-arrival time, enabling fine-grain bandwidth allocation. In an IaaS system, MITTS enables Cloud customers to express their memory distribution needs and pay commensurately. For instance, MITTS enables charging customers that have bursty memory traffic more than customers with uniform memory traffic for the same aggregate bandwidth. Beyond IaaS systems, MITTS can also be used to optimize for throughput or fairness in a general purpose multi-program workload. MITTS uses an online genetic algorithm to configure hardware bins, which can adapt for program phases and variable input sets. We have implemented MITTS in Verilog and have taped-out the design in a 25-core 32nm processor and find that MITTS requires less than 0.9% of core area. We evaluate across SPECint, PARSEC, Apache, and bhm Mail Server workloads, and find that MITTS achieves an average 1.18x performance gain compared to the best static bandwidth allocation, a 2.69x average performance/cost advantage in an IaaS setting, and up to 1.17x better throughput and 1.52x better fairness when compared to conventional memory bandwidth provisioning techniques.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages532-544
Number of pages13
ISBN (Electronic)9781467389471
DOIs
StatePublished - Aug 24 2016
Event43rd International Symposium on Computer Architecture, ISCA 2016 - Seoul, Korea, Republic of
Duration: Jun 18 2016Jun 22 2016

Publication series

NameProceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016

Other

Other43rd International Symposium on Computer Architecture, ISCA 2016
CountryKorea, Republic of
CitySeoul
Period6/18/166/22/16

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Keywords

  • IaaS Clouds
  • Memory Architecture
  • Memory Management
  • Multicore

Fingerprint Dive into the research topics of 'MITTS: Memory Inter-arrival Time Traffic Shaping'. Together they form a unique fingerprint.

  • Cite this

    Zhou, Y., & Wentzlaff, D. (2016). MITTS: Memory Inter-arrival Time Traffic Shaping. In Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016 (pp. 532-544). [7551420] (Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCA.2016.53