PC2A: Predicting Collective Contextual Anomalies via LSTM With Deep Generative Model

Shaoyu Dou, Kai Yang, H. Vincent Poor

Research output: Contribution to journalArticle

Abstract

Proactive anomaly detection and diagnosis play an essential role in ensuring the security and stability of a large-scale information technology (IT) system with thousands or even millions of components that are interacting with each other. Collective contextual anomalies (CCAs) carry the characteristics of both collective and contextual anomalies. This type of anomalies is common in IT system monitoring, often manifested as security risks to be ameliorated, service outages to be eliminated, or stragglers to be mitigated. However, most existing studies emphasize primarily on the detection of point anomalies while the prediction or early detection of CCA has been an underexplored topic. In this paper, we propose a framework for discovering and studying CCAs in multiple time series based on a combination of semi-supervised deep learning, time series modeling, and graph analysis. A primary advantage of the proposed framework is that it can effectively predict CCAs with no human intervention. In addition, the performance of the proposed method can be further enhanced via learning from a small amount of anomalous training data, if it is available. Finally, the proposed framework is of low computational complexity and is thus particularly suitable for large-scale data streams. Simulation studies are carried out to reveal the superior performance of the proposed method and underscore the significant benefits of combining deep neural networks with time series analysis and graph models for the prediction and analysis of CCAs.

Original languageEnglish (US)
Article number8794857
Pages (from-to)9645-9655
Number of pages11
JournalIEEE Internet of Things Journal
Volume6
Issue number6
DOIs
StatePublished - Dec 2019

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Keywords

  • Anomaly prediction
  • LSTM neural-network
  • collective anomaly
  • cyber security
  • deep generative model
  • high-dimensional time series
  • root cause analysis

Fingerprint Dive into the research topics of 'PC<sup>2</sup>A: Predicting Collective Contextual Anomalies via LSTM With Deep Generative Model'. Together they form a unique fingerprint.

  • Cite this