Graph Sampling and GPU Kernel Co-Design for Scalable GNN Training

Abstract

Graph neural networks have become a central computational approach for learning over relational data, but their training pipelines remain difficult to scale because they combine irregular memory access, rapidly expanding neighborhoods, and tightly coupled sampling and execution stages. This paper examines graph sampling and GPU kernel co-design as an integrated systems strategy for scalable graph neural network training. Rather than treating sampling as a preprocessing utility or GPU kernels as isolated optimization targets, the analysis argues that scalability emerges from the joint design of graph traversal policies, memory hierarchies, workload balancing, and deployment governance. The paper first reviews foundational graph neural network architectures and identifies the structural bottlenecks that arise when neighborhood expansion is mapped onto parallel hardware. It then evaluates major sampling families, including layer-wise importance sampling, clustering, and subgraph sampling, with attention to their statistical consequences and infrastructure costs. The discussion next turns to GPU kernel co-design, considering sparse aggregation, feature gathering, fusion opportunities, and memory-centric scheduling. It also explores how distributed deployment, multi-tenant serving, and sustainability constraints reshape the design space. Robustness and fairness are treated as system-level properties rather than post hoc corrections, because sampling policies and kernel scheduling can amplify or suppress particular regions of a graph. The paper concludes that scalable graph neural network training requires an interdisciplinary architecture in which algorithmic sampling, hardware-aware execution, and institutional governance co-evolve. Such co-design can improve throughput and energy efficiency while also making training behavior more predictable, auditable, and equitable.