TY - GEN
T1 - F4
T2 - 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014
AU - Muralidhar, Subramanian
AU - Lloyd, Wyatt
AU - Roy, Sabyasachi
AU - Hill, Cory
AU - Lin, Ernest
AU - Liu, Weiwen
AU - Pan, Satadru
AU - Shankar, Shiva
AU - Sivakumar, Viswanath
AU - Tang, Linpeng
AU - Kumar, Sanjeev
PY - 2014/1/1
Y1 - 2014/1/1
N2 - Facebook's corpus of photos, videos, and other Binary Large OBjects (BLOBs) that need to be reliably stored and quickly accessible is massive and continues to grow. As the footprint of BLOBs increases, storing them in our traditional storage system, Haystack, is becoming increasingly inefficient. To increase our storage efficiency, measured in the effective-replication-factor of BLOBs, we examine the underlying access patterns of BLOBs and identify temperature zones that include hot BLOBs that are accessed frequently and warm BLOBs that are accessed far less often. Our overall BLOB storage system is designed to isolate warm BLOBs and enable us to use a specialized warm BLOB storage system, f4. f4 is a new system that lowers the effective-replication-factor of warm BLOBs while remaining fault tolerant and able to support the lower throughput demands. f4 currently stores over 65PBs of logical BLOBs and reduces their effective-replication-factor from 3.6 to either 2.8 or 2.1. f4 provides low latency; is resilient to disk, host, rack, and datacenter failures; and provides sufficient throughput for warm BLOBs.
AB - Facebook's corpus of photos, videos, and other Binary Large OBjects (BLOBs) that need to be reliably stored and quickly accessible is massive and continues to grow. As the footprint of BLOBs increases, storing them in our traditional storage system, Haystack, is becoming increasingly inefficient. To increase our storage efficiency, measured in the effective-replication-factor of BLOBs, we examine the underlying access patterns of BLOBs and identify temperature zones that include hot BLOBs that are accessed frequently and warm BLOBs that are accessed far less often. Our overall BLOB storage system is designed to isolate warm BLOBs and enable us to use a specialized warm BLOB storage system, f4. f4 is a new system that lowers the effective-replication-factor of warm BLOBs while remaining fault tolerant and able to support the lower throughput demands. f4 currently stores over 65PBs of logical BLOBs and reduces their effective-replication-factor from 3.6 to either 2.8 or 2.1. f4 provides low latency; is resilient to disk, host, rack, and datacenter failures; and provides sufficient throughput for warm BLOBs.
UR - http://www.scopus.com/inward/record.url?scp=85075667407&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85075667407&partnerID=8YFLogxK
M3 - Conference contribution
T3 - Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014
SP - 383
EP - 398
BT - Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014
PB - USENIX Association
Y2 - 6 October 2014 through 8 October 2014
ER -