Dynamic Graph Neural Network Acceleration with Runtime-Aware GPU Kernels
Keywords:
dynamic graph neural networks, GPU acceleration, runtime systems, kernel specialization, graph systems, sustainable computing, fairness, governanceAbstract
Dynamic graph neural networks have become a central computational abstraction for modeling relational systems whose entities, edges, and features change over time. Although early graph neural network research established strong algorithmic foundations for representation learning on static graphs, the acceleration of dynamic graph workloads on graphics processing units remains constrained by irregular memory access, rapidly shifting sparsity patterns, temporal locality, and workload imbalance. This paper examines runtime-aware GPU kernel acceleration as a system-level response to those constraints. Rather than treating kernel optimization as a purely local performance problem, the discussion frames runtime-aware acceleration as an architectural and infrastructural practice that couples graph partitioning, memory hierarchy management, scheduling, precision selection, and temporal windowing. The paper analyzes structural trade-offs among throughput, latency, energy use, portability, and robustness, and it considers how deployment decisions influence fairness, accountability, and policy outcomes in socio-technical infrastructures. Through cross-domain comparisons involving transportation, finance, public health, and communication networks, the paper argues that dynamic graph neural network acceleration must be understood as a governance-sensitive infrastructure problem. The conclusion emphasizes that sustainable acceleration requires co-design across algorithms, runtimes, hardware, and institutional oversight, and that performance claims should be evaluated alongside reproducibility, energy intensity, and equitable access to computational capacity.
References
1. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., & Monfardini, G. (2009). The graph neural network model. IEEE Transactions on Neural Networks, 20(1), 61–80. https://doi.org/10.1109/TNN.2008.2005605
2. Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations.
3. Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30 (pp. 1024–1034). Curran Associates.
4. Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4–24. https://doi.org/10.1109/TNNLS.2020.2978386
5. Peng, H., Xie, X., Shivdikar, K., Hasan, M. A., Zhao, J., Huang, S., ... & Ding, C. (2024, April). Maxk-gnn: Extremely fast gpu kernel design for accelerating graph neural networks training. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (pp. 683-698).
6. Fey, M., & Lenssen, J. E. (2019). Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds.
7. Wang, M., Yu, L., Zheng, D., Gan, Q., Gai, Y., Ye, Z., ... & Zhou, J. (2019). Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv. https://arxiv.org/abs/1909.01315
8. Rozemberczki, B., Scherer, P., He, Y., Panagopoulos, G., Riedel, A., Astefanoaei, M., ... & Sarkar, R. (2021). PyTorch Geometric Temporal: Spatiotemporal signal processing with neural machine learning models. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (pp. 4564–4573). https://doi.org/10.1145/3459637.3482014
9. Pareja, A., Domeniconi, G., Chen, J., Ma, T., Suzumura, T., Kanezashi, H., ... & Leiserson, C. (2020). EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, 34(04), 5363–5370. https://doi.org/10.1609/aaai.v34i04.5984
10. Xu, D., Ruan, C., Korpeoglu, E., Kumar, S., & Achan, K. (2020). Inductive representation learning on temporal graphs. In Proceedings of the 8th International Conference on Learning Representations.
11. Rossi, E., Chamberlain, B., Frasca, F., Eynard, D., Monti, F., & Bronstein, M. (2020). Temporal graph networks for deep learning on dynamic graphs. arXiv. https://arxiv.org/abs/2006.10637
12. Wang, Y., Feng, B., Li, G., Xie, S., Li, Y., Ding, Y., & Xie, Y. (2021). GNNAdvisor: An adaptive and efficient runtime system for GNN acceleration on GPUs. In Proceedings of the 15th USENIX Symposium on Operating Systems Design and Implementation (pp. 515–531).
13. Jia, Z., Lin, S., Ying, R., You, J., Leskovec, J., & Aiken, A. (2020). Redundancy-free computation for graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 997–1005). https://doi.org/10.1145/3394486.3403142
14. Geng, T., Li, A., Shi, R., Wu, C., Wang, T., Li, Y., ... & Herbordt, M. C. (2020). AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In Proceedings of the 53rd IEEE/ACM International Symposium on Microarchitecture (pp. 922–936). https://doi.org/10.1109/MICRO50266.2020.00079
15. Yan, M., Deng, L., Hu, X., Liang, L., Feng, Y., Ma, X., ... & Xie, Y. (2020). HyGCN: A GCN accelerator with hybrid architecture. In Proceedings of the 26th IEEE International Symposium on High Performance Computer Architecture (pp. 15–29). https://doi.org/10.1109/HPCA47549.2020.00012
16. Abadal, S., Jain, A., Guirado, R., López-Alonso, J., & Alarcón, E. (2021). Computing graph neural networks: A survey from algorithms to accelerators. ACM Computing Surveys, 54(9), 1–38. https://doi.org/10.1145/3477141
17. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), 1–35. https://doi.org/10.1145/3457607
18. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 3645–3650). https://doi.org/10.18653/v1/P19-1355
19. Mittelstadt, B. (2019). Principles alone cannot guarantee ethical AI. Nature Machine Intelligence, 1(11), 501–507. https://doi.org/10.1038/s42256-019-0114-4
20. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., ... & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 33–44). https://doi.org/10.1145/3351095.3372873
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Artificial Intelligence and Machine Learning Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.