Do We Really Need Complicated Model Architectures For Temporal Networks?

Recurrent neural network (RNN) and self-attention mechanism (SAM) are the de facto methods to extract spatial-temporal information for temporal graph learning. Interestingly, we found that although both RNN and SAM could lead to a good performance, in practice neither of them is always necessary. In...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Cong, Weilin, Zhang, Si, Kang, Jian, Yuan, Baichuan, Wu, Hao, Zhou, Xin, Tong, Hanghang, Mahdavi, Mehrdad
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 22.02.2023
Subjects
Coders
Multilayer perceptrons
Multilayers
Recurrent neural networks
Online AccessGet full text

Cover

More Information
Summary:Recurrent neural network (RNN) and self-attention mechanism (SAM) are the de facto methods to extract spatial-temporal information for temporal graph learning. Interestingly, we found that although both RNN and SAM could lead to a good performance, in practice neither of them is always necessary. In this paper, we propose GraphMixer, a conceptually and technically simple architecture that consists of three components: (1) a link-encoder that is only based on multi-layer perceptrons (MLP) to summarize the information from temporal links, (2) a node-encoder that is only based on neighbor mean-pooling to summarize node information, and (3) an MLP-based link classifier that performs link prediction based on the outputs of the encoders. Despite its simplicity, GraphMixer attains an outstanding performance on temporal link prediction benchmarks with faster convergence and better generalization performance. These results motivate us to rethink the importance of simpler model architecture.
ISSN:2331-8422