MPGraf: a Modular and Pre-trained Graphformer for Learning to Rank at Web-scale

• Published in: Proceedings (IEEE International Conference on Data Mining) pp. 339 - 348
• Main Authors: Li, Yuchen, Xiong, Haoyi, Kong, Linghe, Sun, Zeyi, Chen, Hongyang, Wang, Shuaiqiang, Yin, Dawei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.12.2023
• Subjects: Bipartite graph; Graph neural networks; Learning to Rank; Modular Network; Pretraining; Search engines; Stacking; Surgery; Task analysis; Transformers; Web Search
• ISSN: 2374-8486
• DOI: 10.1109/ICDM58522.2023.00043

Both Transformer and Graph Neural Networks (GNNs) have been used in learning to rank (LTR), they however adhere to two distinct-yet-complementary problem formulations, i.e., ranking score regression based on query-webpage pairs and link prediction within query-webpage bipartite graphs, respectively. Though it is possible to pre-train GNNs or Transformers on source datasets and fine-tune them subject to sparsely annotated LTR datasets separately, the source-target distribution shifts across the pairs and bipartite graphs domains make it extremely difficult to integrate these diverse models into a single LTR framework at a web-scale. We introduce the novel MPGraf model, which utilizes a modular and capsule-based pre-training approach, aiming to incorporate regression capacities from Transformers and link prediction capabilities of GNNs cohesively. Specifically, rather than simply combining these two modules in stacking or parallelizing architectures, MPGraf proposes a three-step method to (1) construct query-webpage graphs from sparsely annotated query-webpage pairs, (2) pre-train the Transformer and GNN modules using source datasets from two domains, and (3) "surgically" fine-tune the integrated models with pretrained weights while handling the distribution shifts of two domains. Extensive experiments have been done to evaluate the performance of MPGraf using real-world datasets collected from large-scale search engines. The results show that MPGraf can outperform baseline algorithms on several major metrics. Further, we deploy and evaluate MPGraf atop a large-scale search engine with realistic web traffics via A/B tests, where we can still observe significant improvement. MPGraf performs consistently in both offline and online evaluations.