PINE: Universal Deep Embedding for Graph Nodes via Partial Permutation Invariant Set Functions

Shupeng Gui, Xiangliang Zhang, Pan Zhong, Shuang Qiu, Mingrui Wu, Jieping Ye, Zhengdao Wang, Ji Liu
2021 IEEE Transactions on Pattern Analysis and Machine Intelligence  
Graph node embedding aims at learning a vector representation for all nodes given a graph. It is a central problem in many machine learning tasks (e.g., node classification, recommendation, community detection). The key problem in graph node embedding lies in how to define the dependence to neighbors. Existing approaches specify (either explicitly or implicitly) certain dependencies on neighbors, which may lead to loss of subtle but important structural information within the graph and other
more » ... endencies among neighbors. This intrigues us to ask the question: can we design a model to give the adaptive flexibility of dependencies to each node's neighborhood. In this paper, we propose a novel graph node embedding method (named PINE) via a novel notion of partial permutation invariant set function, to capture any possible dependence. Our method 1) can learn an arbitrary form of the representation function from the neighborhood, without losing any potential dependence structures, and 2) is applicable to both homogeneous and heterogeneous graph embedding, the latter of which is challenged by the diversity of node types. Furthermore, we provide theoretical guarantee for the representation capability of our method for general homogeneous and heterogeneous graphs. Empirical evaluation results on benchmark data sets show that our proposed PINE method outperforms the state-of-the-art approaches on producing node vectors for various learning tasks of both homogeneous and heterogeneous graphs. Index Terms-Graph embedding, partial permutation invariant set function, representation learning ! • S. Gui is with University of Rochester (, X. Zhang is with King is currently the head of AI platform department and Seattle AI lab at Kwai Inc. He is also an affiliated professor at University of Rochester. He graduated from University of Wisconsin-Madison in computer science. He worked in machine learning and artificial intelligence for more than 15 years. His research covers a wide range of ML and AI areas, such as optimization, computer vision, reinforcement learning, recommendation system, bioinformatics, robotics, etc. He received the IBM faculty award in 2017 and a few best paper or nomination awards in top tier CS conferences such as UAI and KDD. He is also an awardee of MIT TR35 China 2017. He serves the ML community as an AC, SPC, or PC for top ML conferences such as AAAI, IJCAI, NIPS, ICML, etc.
doi:10.1109/tpami.2021.3061162 fatcat:jgmhyduzvfel3ljpkdigldwzdq