A framework of graph classification with self-supervised heterogeneous graph neural network

doi:10.7523/j.ucas.2023.048

Abstract

Abstract: Graph data exists widely in various forms, and the tasks of graph classification have great significance for many problems. However, the tasks of graph classification are still faced with many challenges, including how to make full use of the semantic information contained in the graph structures, and how to further reduce the computational complexity and the cost of obtaining labels. In this paper, a construction method of hyper-node heterogeneous network is proposed for the first time, and a new framework GChgnn which can be applied to graph classification is proposed. The GChgnn framework achieves the following goals through the introduction of double-view graph representation mechanism and self-supervised contrastive learning: ① measure the similarity between the objectives of the large-scale graph classification tasks; ② inspired by the graph matching methods, improve the accuracy of similarity measurement by the cross-graph idea, and make up for the lack of the explicit expression of graph embedding; ③ avoid designing complicated convolution and pooling operators in the network. After testing on some public datasets, the framework outperforms other existing methods.

Key words: graph classification, heterogeneous graph neutral network, self-supervised learning, contrastive learning

CLC Number:

TP389.1

YUAN Ming, ZHAO Tong, . A framework of graph classification with self-supervised heterogeneous graph neural network[J]. Journal of University of Chinese Academy of Sciences, DOI: 10.7523/j.ucas.2023.048.

References

[1] Wang X, Ji H Y, Shi C, et al.Heterogeneous graph attention network[C]//WWW '19: The World Wide Web Conference. May 13-17, 2019, San Francisco, CA, USA. New York: ACM, 2019: 2022-2032. DOI: 10.1145/3308558.3313562.
[2] Kriege N M, Johansson F D, Morris C.A survey on graph kernels[J]. Applied Network Science, 2020, 5(1): 1-42. DOI: 10.1007/s41109-019-0195-3.
[3] Nikolentzos G, Siglidis G, Vazirgiannis M.Graph kernels: a survey[J]. Journal of Artificial Intelligence Research, 2021, 72: 943-1027. DOI: 10.1613/jair.1.13225.
[4] Foggia P, Percannella G, Vento M.Graph matching and learning in pattern recognition in the last 10 years[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2014, 28(1): 1450001. DOI: 10.1142/S0218001414500013.
[5] Ma Y, Wang S H, Aggarwal C C, et al.Graph convolutional networks with EigenPooling[C]// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. August 4 - 8, 2019, Anchorage, AK, USA. New York: ACM, 2019: 723-731. DOI: 10.1145/3292500.3330982.
[6] Ying R, You J X, Morris C, et al.Hierarchical graph representation learning with differentiable pooling[C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. December 3 - 8, 2018, Montréal, Canada. New York: ACM, 2018: 4805-4815. DOI: 10.48550/arXiv.1806.08804.
[7] Khasahmadi A H, Hassani K, Moradi P, et al. Memory-based graph networks[EB/OL].542020: arXiv:2002.09518.(2020-02-21)[ 2023-03-06 ] https://arxiv.org/abs/2002.09518.
[8] Lee J, Lee I, Kang J.Self-attention graph pooling[C]//International Conference on Machine Learning. PMLR, 2019: 3734-3743. DOI: 10.48550/arXiv.1904.08082.
[9] Shawe-Taylor J, Cristianini N.Kernel methods for pattern analysis[M]. Cambridge, UK: Cambridge University Press, 2004. DOI: 10.1017/CBO9780511809682.
[10] Haussler D.Convolution kernels on discrete structures[R]. Technical report, Department of Computer Science, University of California at Santa Cruz, 1999.
[11] Borgwardt K M, Kriegel H P. Shortest-path kernels on graphs[C]//Fifth IEEE International Conference on Data Mining (ICDM'05). November 27-30, 2005, Houston, TX, USA. IEEE,2006: 8pp. DOI: 10.1109/ICDM.2005.132.
[12] Shervashidze N, Vishwanathan S V N, Petri T, et al. Efficient graphlet kernels for large graph comparison[C]//Artificial Intelligence and Statistics. PMLR, 2009: 488-495.
[13] Shervashidze N, Schweitzer P, Van Leeuwen E J, et al. Weisfeiler-lehman graph kernels[J]. Journal of Machine Learning Research, 2011, 12: 2539-2561.
[14] Yanardag P, Vishwanathan S V N. Deep graph kernels[C]//Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. August 10 - 13, 2015, Sydney, NSW, Australia. New York: ACM, 2015: 1365-1374. DOI: 10.1145/2783258.2783417.
[15] Li Y J, Gu C J, Dullien T, et al. Graph matching networks for learning the similarity of graph structured objects[EB/OL].2019: arXiv: 1904.12787.(2019-04-29)[ 2023-03-06 ]https://arxiv.org/abs/1904.12787.
[16] Bai Y S, Ding H, Bian S, et al.SimGNN: A neural network approach to fast graph similarity computation[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. February 11 - 15, 2019, Melbourne VIC, Australia. New York: ACM, 2019: 384-392. DOI: 10.1145/3289600.3290967.
[17] Gilmer J, Schoenholz S S, Riley P F, et al.Neural message passing for Quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning - Volume 70. August 6 - 11, 2017, Sydney, NSW, Australia. New York: ACM, 2017: 1263-1272. DOI: 10.48550/arXiv.1704.01212.
[18] Zhang M H, Cui Z C, Neumann M, et al.An end-to-end deep learning architecture for graph classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1). DOI: 10.1609/aaai.v32i1.11782.
[19] Morris C, Ritzert M, Fey M, et al.Weisfeiler and leman go neural: Higher-order graph neural networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 4602-4609. DOI: 10.1609/aaai.v33i01.33014602.
[20] Niepert M, Ahmed M, Kutzkov K.Learning convolutional neural networks for graphs[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. June 19 - 24, 2016, New York, NY, USA. New York: ACM, 2016: 2014-2023. DOI: 10.48550/arXiv.1605.05273.
[21] Tzeng R C, Wu S H.Ego-CNN: Distributed, Egocentric Representations of Graphs for Detecting Critical Structures[C]//Proceedings of the 36th International Conference on Machine Learning, 2019. DOI: 10.48550/arXiv.1906.09602.
[22] Sun Y Z, Han J W.Meta-path-based search and mining in heterogeneous information networks[J]. Tsinghua Science and Technology, 2013, 18(4): 329-338. DOI: 10.1109/TST.2013.6574671.
[23] Debnath A K, Lopez de Compadre R L, Debnath G, et al. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. Correlation with molecular orbital energies and hydrophobicity[J]. Journal of Medicinal Chemistry, 1991, 34(2): 786-797. DOI: 10.1021/jm00106a046.
[24] Wang X, Lu Y F, Shi C, et al.Dynamic heterogeneous information network embedding with meta-path based proximity[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(3): 1117-1132. DOI: 10.1109/TKDE.2020.2993870.
[25] Wale N, Watson I A, Karypis G.Comparison of descriptor spaces for chemical compound retrieval and classification[J]. Knowledge and Information Systems, 2008, 14(3): 347-375. DOI: 10.1007/s10115-007-0103-5.
[26] Borgwardt K M, Ong C S, Schönauer S, et al.Protein function prediction via graph kernels[J]. Bioinformatics, 2005, 21(suppl_1): i47-i56. DOI: 10.1093/bioinformatics/bti1007.
[27] Narayanan A, Chandramohan M, Chen L H, et al. subgraph2vec: learning distributed representations of rooted sub-graphs from large graphs[EB/OL].2016: arXiv: 1606.08928.(2016-06-29)[ 2023-03-06 ] https://arxiv.org/abs/1606.08928.
[28] Kondor R, Pan H.The multiscale Laplacian graph kernel[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. December 5 - 10, 2016, Barcelona, Spain. New York: ACM, 2016: 2990-2998. DOI: 10.48550/arXiv.1603.06186.
[29] Dai H J, Dai B, Song L.Discriminative embeddings of latent variable models for structured data[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48. June 19 - 24, 2016, New York, NY, USA. New York: ACM, 2016: 2702-2711. DOI: 10.48550/arXiv.1603.05629.
[30] Atwood J, Towsley D.Diffusion-convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2016, 29. DOI: 10.48550/arXiv.1511.02136.
[31] Veličković P, Cucurull G, Casanova A, et al.Graph attention networks[J]. International Conference On Learning Representations, 2018. DOI: 10.48550/arXiv.1710.10903.
[32] Peng H, Li J X, Gong Q R, et al.Motif-matching based subgraph-level attentional convolutional network for graph classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(4): 5387-5394. DOI: 10.1609/aaai.v34i04.5987.
[33] Lu Y F, Shi C, Hu L M, et al.Relation structure-aware heterogeneous information network embedding[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 4456-4463. DOI: 10.1609/aaai.v33i01.33014456.