Shark: Scaling file servers via cooperative caching

Siddhartha Annapureddy, Michael J. Freedman, David Mazières

Research output: Contribution to conferencePaperpeer-review

95 Scopus citations

Abstract

Network file systems offer a powerful, transparent interface for accessing remote data. Unfortunately, in current network file systems like NFS, clients fetch data from a central file server, inherently limiting the system's ability to scale to many clients. While recent distributed (peer-to-peer) systems have managed to eliminate this scalability bottleneck, they are often exceedingly complex and provide non-standard models for administration and accountability. We present Shark, a novel system that retains the best of both worlds-the scalability of distributed systems with the simplicity of central servers. Shark is a distributed file system designed for large-scale, wide-area deployment, while also providing a drop-in replacement for local-area file systems. Shark introduces a novel cooperative-caching mechanism, in which mutually-distrustful clients can exploit each others' file caches to reduce load on an origin file server. Using a distributed index, Shark clients find nearby copies of data, even when files originate from different servers. Performance results show that Shark can greatly reduce server load and improve client latency for read-heavy workloads both in the wide and local areas, while still remaining competitive for single clients in the local area. Thus, Shark enables modestly-provisioned file servers to scale to hundreds of read-mostly clients while retaining traditional usability, consistency, security, and accountability.

Original languageEnglish (US)
Pages129-142
Number of pages14
StatePublished - 2005
Externally publishedYes
Event2nd Symposium on Networked Systems Design and Implementation, NSDI 2005 - Boston, United States
Duration: May 2 2005May 4 2005

Conference

Conference2nd Symposium on Networked Systems Design and Implementation, NSDI 2005
CountryUnited States
CityBoston
Period5/2/055/4/05

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Shark: Scaling file servers via cooperative caching'. Together they form a unique fingerprint.

Cite this