An experimental study of balance in matrix factorization

Jennifer Hsia, Peter J. Ramadge

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We experimentally examine how gradient descent navigates the landscape of matrix factorization to obtain a global minimum. First, we review the critical points of matrix factorization and introduce a balanced factorization. By focusing on the balanced critical point at the origin and a subspace of unbalanced critical points, we study the effect of balance on gradient descent, including an initially unbalanced factorization and adding a balance-regularizer to the objective in the MF problem. Simulations demonstrate that maintaining a balanced factorization enables faster escape from saddle points and overall faster convergence to a global minimum.

Original languageEnglish (US)
Title of host publication2021 55th Annual Conference on Information Sciences and Systems, CISS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665412681
DOIs
StatePublished - Mar 24 2021
Event55th Annual Conference on Information Sciences and Systems, CISS 2021 - Baltimore, United States
Duration: Mar 24 2021Mar 26 2021

Publication series

Name2021 55th Annual Conference on Information Sciences and Systems, CISS 2021

Conference

Conference55th Annual Conference on Information Sciences and Systems, CISS 2021
CountryUnited States
CityBaltimore
Period3/24/213/26/21

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Information Systems
  • Information Systems and Management

Keywords

  • Balance
  • Gradient descent
  • Matrix factorization
  • Non-convex optimization
  • Saddle points

Fingerprint Dive into the research topics of 'An experimental study of balance in matrix factorization'. Together they form a unique fingerprint.

Cite this