Space-alternating attribute-distributed sparse learning

Dmitriy Shutin, Haipeng Zheng, Bernard H. Fleury, Sanjeev R. Kulkarni, H. Vincent Poor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

The paper proposes a new variational Bayesian algorithm for multivariate regression with attribute-distributed or dimensionally distributed data. Compared to the existing approaches the proposed algorithm exploits the variational version of the Space-Alternating Generalized Expectation- Maximization (SAGE) algorithm that by means of admissible hidden data - an analog of the complete data in the EM framework-allows parameters of a single agent to be updated assuming that parameters of the other agents are fixed. This allows learning to be implemented in a distributed fashion by sequentially updating the agents one after another. Inspired by Bayesian sparsity techniques, the algorithm also introduces constraints on the agent parameters via parametric priors. This adds a mechanism for pruning irrelevant agents, as well as for minimizing the effect of overfitting. Using synthetic data, as well as measurement data from the UCI Machine Learning Repository it is demonstrated that the proposed algorithm outperforms existing solutions both in the achieved mean-square error (MSE), as well as in convergence speed due to the ability to sparsify noninformative agents, while at the same time allowing distributed implementation and flexible agent update protocols.

Original languageEnglish (US)
Title of host publication2010 2nd International Workshop on Cognitive Information Processing, CIP2010
Pages209-214
Number of pages6
DOIs
StatePublished - Nov 22 2010
Event2010 2nd International Workshop on Cognitive Information Processing, CIP2010 - Elba Island, Italy
Duration: Jun 14 2010Jun 16 2010

Publication series

Name2010 2nd International Workshop on Cognitive Information Processing, CIP2010

Other

Other2010 2nd International Workshop on Cognitive Information Processing, CIP2010
CountryItaly
CityElba Island
Period6/14/106/16/10

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Information Systems

Fingerprint Dive into the research topics of 'Space-alternating attribute-distributed sparse learning'. Together they form a unique fingerprint.

  • Cite this

    Shutin, D., Zheng, H., Fleury, B. H., Kulkarni, S. R., & Poor, H. V. (2010). Space-alternating attribute-distributed sparse learning. In 2010 2nd International Workshop on Cognitive Information Processing, CIP2010 (pp. 209-214). [5604254] (2010 2nd International Workshop on Cognitive Information Processing, CIP2010). https://doi.org/10.1109/CIP.2010.5604254