Attribute-distributed learning: The iterative covariance optimization algorithm and its applications

This paper introduces a framework for multivariate regression with attribute-distributed data on a distributed system with a fusion center. Unlike other types of algorithms for attribute-distributed learning that directly refit the ensemble residual or average among the predictions of the agents, the new algorithm, the iterative covariance optimization algorithm (ICOA), coordinates the agents to reshape the covariance matrix of the individual training residuals so that the ensemble estimator, a linear combination of the individual estimators, minimizes the ensemble training error. Moreover, ICOA empirically demonstrates strong insusceptibility to overtraining, especially compared with residual refitting algorithms. Extensive simulations on both artificial and real datasets indicate that ICOA consistently outperforms weighted averaging algorithms and residual refitting algorithms.