Sensitivity of PCA for traffic anomaly detection

Haakon Ringberg, Augustin Soule, Jennifer L. Rexford, Christophe Diot

Research output: Chapter in Book/Report/Conference proceedingConference contribution

171 Scopus citations

Abstract

Detecting anomalous traffic is a crucial part of managing IP networks. In recent years, network-wide anomaly detection based on Principal Component Analysis (PCA) has emerged as a powerful method for detecting a wide variety of anomalies. We show that tuning PCA to operate effectively in practice is difficult and requires more robust techniques than have been presented thus far. We analyze a week of network-wide traffic measurements from two IP backbones (Abilene and Geant) across three different traffic aggregations (ingress routers, OD flows, and input links), and conduct a detailed inspection of the feature time series for each suspected anomaly. Our study identifies and evaluates four main challenges of using PCA to detect traffic anomalies: (i) the false positive rate is very sensitive to small differences in the number of principal components in the normal subspace, (ii) the effectiveness of PCA is sensitive to the level of aggregation of the traffic measurements, (iii) a large anomaly may in advertently pollute the normal subspace, (iv) correctly identifying which flow triggered the anomaly detector is an inherently challenging problem.

Original languageEnglish (US)
Title of host publicationSIGMETRICS'07 - Proceedings of the 2007 International Conference on Measurement and Modeling of Computer Systems
Pages109-120
Number of pages12
Edition1
DOIs
StatePublished - Nov 29 2007
EventSIGMETRICS'07 - 2007 International Conference on Measurement and Modeling of Computer Systems - San Diego, CA, United States
Duration: Jun 12 2007Jun 16 2007

Publication series

NamePerformance Evaluation Review
Number1
Volume35
ISSN (Print)0163-5999

Other

OtherSIGMETRICS'07 - 2007 International Conference on Measurement and Modeling of Computer Systems
CountryUnited States
CitySan Diego, CA
Period6/12/076/16/07

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Sensitivity of PCA for traffic anomaly detection'. Together they form a unique fingerprint.

  • Cite this

    Ringberg, H., Soule, A., Rexford, J. L., & Diot, C. (2007). Sensitivity of PCA for traffic anomaly detection. In SIGMETRICS'07 - Proceedings of the 2007 International Conference on Measurement and Modeling of Computer Systems (1 ed., pp. 109-120). (Performance Evaluation Review; Vol. 35, No. 1). https://doi.org/10.1145/1269899.1254895