EigenGame: PCA as a Nash Equilibrium [article]

Ian Gemp, Brian McWilliams, Claire Vernade, Thore Graepel
2021 arXiv   pre-print
We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm -- which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization -- is naturally decentralized and hence parallelizable through message passing. We
more » ... rate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.
arXiv:2010.00554v2 fatcat:fjdmz2a5ibfxnbayse7jh2wlii