Graph Learning by Dynamic Sampling

Hermes, Luca; Liuliakov, Aleksei; Schilling, Malte

Abstract

Graph neural networks based on message-passing rely on the principle of neighborhood aggregation which has shown to work well for many graph tasks. In other cases these approaches appear insufficient, for example, when graphs are heterophilic. In such cases, it can help to modulate the aggregation method depending on the characteristic of the current neighborhood. Furthermore, when considering higher-order relations, heterophilic settings become even more important. In this work, we investigate a sparse version of message-passing that allows selective neighbor integration and aims for learning to identify most salient nodes that are then integrated over. In our approach, information on individual nodes is encoded by generating distinct walks. Because these walks follow distinct trajectories, the higher-order neighborhood grows only linearly which mitigates information bottlenecks. Overall, we aim to find the most salient substructures by deploying a learnable sampling strategy. We validate our method on commonly used graph benchmarks and show the effectiveness especially in heterophilic graphs. We finally discuss possible extensions to the framework.