NAAM: Node-Aware Attention Mechanism for Distilling GNNs-to-MLP (Student Abstract)

Abstract Recently, researchers have focused on methods that not only distill knowledge from a Graph Neural Network (GNN) into a Multi-Layer Perceptron (MLP) but also leverage multiple teacher GNNs. However, existing methods assign a single attention weight to each teacher GNN. We propose a NodeAware Attention Mechanism (NAAM) that flexibly adjusts the attention weight for each node to leverage multiple GNNs fully. Experimental results show that NAAM outperforms existing GNN-to-MLP methods. our source code is available at: https://github.com/NakayamaItsuki/NAAM.