TY - GEN
T1 - Generalization and equilibrium in generative adversarial nets (GANs)
AU - Arora, Sanjeev
AU - Ge, Rong
AU - Liang, Yingyu
AU - Ma, Tengyu
AU - Zhang, Yi
N1 - Publisher Copyright:
© 2017 International Machine Learning Society (IMLS). All rights reserved.
PY - 2017
Y1 - 2017
N2 - Generalization is defined training of generative adversarial network (GAN), and it's shown that generalization is not guaranteed for the popular distances between distributions such as Jensen-Shannon or Wasserstein. In particular, training may appear to be successful and yet the trained distribution may be arbitrarily far from the target distribution in standard metrics. It is shown that generalization does occur for a much weaker metric we call neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a natural training objective (Wasserstein) when generator capacity and training set sizes are moderate. Finally, the above theoretical ideas suggest a new training protocol, mix+GAN, which can be combined with any existing method, and empirically is found to improves some existing GAN protocols out of the box.
AB - Generalization is defined training of generative adversarial network (GAN), and it's shown that generalization is not guaranteed for the popular distances between distributions such as Jensen-Shannon or Wasserstein. In particular, training may appear to be successful and yet the trained distribution may be arbitrarily far from the target distribution in standard metrics. It is shown that generalization does occur for a much weaker metric we call neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a natural training objective (Wasserstein) when generator capacity and training set sizes are moderate. Finally, the above theoretical ideas suggest a new training protocol, mix+GAN, which can be combined with any existing method, and empirically is found to improves some existing GAN protocols out of the box.
UR - http://www.scopus.com/inward/record.url?scp=85048679392&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85048679392&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85048679392
T3 - 34th International Conference on Machine Learning, ICML 2017
SP - 322
EP - 349
BT - 34th International Conference on Machine Learning, ICML 2017
PB - International Machine Learning Society (IMLS)
T2 - 34th International Conference on Machine Learning, ICML 2017
Y2 - 6 August 2017 through 11 August 2017
ER -