SAQL: A stream-based query system for real-time abnormal system behavior detection

Peng Gao, Xusheng Xiao, Ding Li, Zhichun Li, Kangkook Jee, Zhenyu Wu, Chung Hwan Kim, Sanjeev R. Kulkarni, Prateek Mittal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

Recently, advanced cyber attacks, which consist of a sequence of steps that involve many vulnerabilities and hosts, compromise the security of many well-protected businesses. This has led to the solutions that ubiquitously monitor system activities in each host (big data) as a series of events, and search for anomalies (abnormal behaviors) for triaging risky events. Since fighting against these attacks is a time-critical mission to prevent further damage, these solutions face challenges in incorporating expert knowledge to perform timely anomaly detection over the large-scale provenance data. To address these challenges, we propose a novel stream-based query system that takes as input, a realtime event feed aggregated from multiple hosts in an enterprise, and provides an anomaly query engine that queries the event feed to identify abnormal behaviors based on the specified anomalies. To facilitate the task of expressing anomalies based on expert knowledge, our system provides a domain-specific query language, SAQL, which allows analysts to express models for (1) rule-based anomalies, (2) time-series anomalies, (3) invariant-based anomalies, and (4) outlier-based anomalies. We deployed our system in NEC Labs America comprising 150 hosts and evaluated it using 1.1TB of real system monitoring data (containing 3.3 billion events). Our evaluations on a broad set of attack behaviors and micro-benchmarks show that our system has a low detection latency (<2s) and a high system throughput (110,000 events/s; supporting ∼4000 hosts), and is more efficient in memory utilization than the existing stream-based complex event processing systems.

Original languageEnglish (US)
Title of host publicationProceedings of the 27th USENIX Security Symposium
PublisherUSENIX Association
Pages639-656
Number of pages18
ISBN (Electronic)9781939133045
StatePublished - Jan 1 2018
Event27th USENIX Security Symposium - Baltimore, United States
Duration: Aug 15 2018Aug 17 2018

Publication series

NameProceedings of the 27th USENIX Security Symposium

Conference

Conference27th USENIX Security Symposium
CountryUnited States
CityBaltimore
Period8/15/188/17/18

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'SAQL: A stream-based query system for real-time abnormal system behavior detection'. Together they form a unique fingerprint.

  • Cite this

    Gao, P., Xiao, X., Li, D., Li, Z., Jee, K., Wu, Z., Kim, C. H., Kulkarni, S. R., & Mittal, P. (2018). SAQL: A stream-based query system for real-time abnormal system behavior detection. In Proceedings of the 27th USENIX Security Symposium (pp. 639-656). (Proceedings of the 27th USENIX Security Symposium). USENIX Association.