SVE: Distributed Video Processing at Facebook Scale

Qi Huang, Petchean Ang, Peter Knowles, Tomasz Nykiel, Iaroslav Tverdokhlib, Amit Yajurvedi, Paul Dapolito, Xifan Yan, Maxim Bykov, Chuen Liang, Mohit Talwar, Abhishek Mathur, Sachin Kulkarni, Matthew Burke, Wyatt Lloyd

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

Videos are an increasingly utilized part of the experience of the billions of people that use Facebook. These videos must be uploaded and processed before they can be shared and downloaded. Uploading and processing videos at our scale, and across our many applications, brings three key requirements: low latency to support interactive applications; a exible programming model for application developers that is simple to program, enables ecient processing, and improves reliability; and robustness to faults and overload. This paper describes the evolution from our initial monolithic encoding script (MES) system to our current Streaming Video Engine (SVE) that overcomes each of the challenges. SVE has been in production since the fall of 2015, provides lower latency than MES, supports many diverse video applications, and has proven to be reliable despite faults and overload.

Original languageEnglish (US)
Title of host publicationSOSP 2017 - Proceedings of the 26th ACM Symposium on Operating Systems Principles
PublisherAssociation for Computing Machinery, Inc
Pages87-103
Number of pages17
ISBN (Electronic)9781450350853
DOIs
StatePublished - Oct 14 2017
Event26th ACM Symposium on Operating Systems Principles, SOSP 2017 - Shanghai, China
Duration: Oct 28 2017Oct 31 2017

Publication series

NameSOSP 2017 - Proceedings of the 26th ACM Symposium on Operating Systems Principles

Other

Other26th ACM Symposium on Operating Systems Principles, SOSP 2017
CountryChina
CityShanghai
Period10/28/1710/31/17

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'SVE: Distributed Video Processing at Facebook Scale'. Together they form a unique fingerprint.

  • Cite this

    Huang, Q., Ang, P., Knowles, P., Nykiel, T., Tverdokhlib, I., Yajurvedi, A., Dapolito, P., Yan, X., Bykov, M., Liang, C., Talwar, M., Mathur, A., Kulkarni, S., Burke, M., & Lloyd, W. (2017). SVE: Distributed Video Processing at Facebook Scale. In SOSP 2017 - Proceedings of the 26th ACM Symposium on Operating Systems Principles (pp. 87-103). (SOSP 2017 - Proceedings of the 26th ACM Symposium on Operating Systems Principles). Association for Computing Machinery, Inc. https://doi.org/10.1145/3132747.3132775