VectorVisor: A Binary Translation Scheme for Throughput-Oriented GPU Acceleration

Samuel Ginzburg, Mohammad Shahrad, Michael J. Freedman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Beyond conventional graphics applications, general-purpose GPU acceleration has had significant impact on machine learning and scientific computing workloads. Yet, it has failed to see widespread use for server-side applications, which we argue is because GPU programming models offer a level of abstraction that is either too low-level (e.g., OpenCL, CUDA) or too high-level (e.g., TensorFlow, Halide), depending on the language. Not all applications fit into either category, resulting in lost opportunities for GPU acceleration. We introduce VectorVisor, a vectorized binary translator that enables new opportunities for GPU acceleration by introducing a novel programming model for GPUs. With VectorVisor, many copies of the same server-side application are run concurrently on the GPU, where VectorVisor mimics the abstractions provided by CPU threads. To achieve this goal, we demonstrate how to (i) provide cross-platform support for system calls and recursion using continuations and (ii) make full use of the excess register file capacity and high memory bandwidth of GPUs. We then demonstrate that our binary translator is able to transparently accelerate certain classes of compute-bound workloads, gaining significant improvements in throughput-per-dollar of up to 2.9× compared to Intel x86-64 VMs in the cloud, and in some cases match the throughput-per-dollar of native CUDA baselines.

Original languageEnglish (US)
Title of host publicationProceedings of the 2023 USENIX Annual Technical Conference, ATC 2023
PublisherUSENIX Association
Pages1017-1037
Number of pages21
ISBN (Electronic)9781939133359
StatePublished - 2023
Event2023 USENIX Annual Technical Conference, ATC 2023 - Boston, United States
Duration: Jul 10 2023Jul 12 2023

Publication series

NameProceedings of the 2023 USENIX Annual Technical Conference, ATC 2023

Conference

Conference2023 USENIX Annual Technical Conference, ATC 2023
Country/TerritoryUnited States
CityBoston
Period7/10/237/12/23

All Science Journal Classification (ASJC) codes

  • General Computer Science

Fingerprint

Dive into the research topics of 'VectorVisor: A Binary Translation Scheme for Throughput-Oriented GPU Acceleration'. Together they form a unique fingerprint.

Cite this