Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data

Changsheng Li, Fan Wei, Weishan Dong, Xiangfeng Wang, Qingshan Liu, Xin Zhang

Research output: Contribution to journalArticlepeer-review

23 Scopus citations

Abstract

Online multiple-output regression is an important machine learning technique for modeling, predicting, and compressing multi-dimensional correlated data streams. In this paper, we propose a novel online multiple-output regression method, called MORES, for streaming data. MORES can dynamically learn the structure of the regression coefficients to facilitate the model's continuous refinement. Considering that limited expressive ability of regression models often leading to residual errors being dependent, MORES intends to dynamically learn and leverage the structure of the residual errors to improve the prediction accuracy. Moreover, we introduce three modified covariance matrices to extract necessary information from all the seen data for training, and set different weights on samples so as to track the data streams' evolving characteristics. Furthermore, an efficient algorithm is designed to optimize the proposed objective function, and an efficient online eigenvalue decomposition algorithm is developed for the modified covariance matrix. Finally, we analyze the convergence of MORES in certain ideal condition. Experiments on two synthetic datasets and three real-world datasets validate the effectiveness and efficiency of MORES. In addition, MORES can process at least 2,000 instances per second (including training and testing) on the three real-world datasets, more than 12 times faster than the state-of-the-art online learning algorithm.

Original languageEnglish (US)
Article number8260965
Pages (from-to)323-336
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume41
Issue number2
DOIs
StatePublished - Feb 1 2019
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Keywords

  • dynamic relationship learning
  • forgetting factor
  • lossless compression
  • Online multiple-output regression

Fingerprint

Dive into the research topics of 'Dynamic Structure Embedded Online Multiple-Output Regression for Streaming Data'. Together they form a unique fingerprint.

Cite this