Skip to content

SC25 Talks at the UMD Booth

SC25 Talks at the UMD Booth

Unmasking Performance Variability in GPU Codes on Production Supercomputers

Performance variability is often a critical issue on GPU-accelerated systems, undermining efficiency and reproducibility. Since large-scale investigations of performance variability on GPU clusters are lacking, we set up a longitudinal experiment on Perlmutter and Frontier. We benchmark representative HPC and AI applications and collect detailed performance data to assess the impact of compute variability, allocated node topology, and network conditions on overall runtime. We also use a ML-based approach to identify potential correlations between these factors and to forecast the execution time. Our analysis identifies network performance as the dominant source of runtime variability. These findings provide crucial insights that can inform the development of future mitigation strategies.

Speaker Bio - Cunyang Wei

Cunyang Wei

Cunyang Wei is a second-year PhD student in the Parallel Software and Systems Group at the University of Maryland, College Park, under the supervision of Professor Abhinav Bhatele. His PhD research focuses on parallel optimization for large-scale GNNs and performance variability. Prior to his PhD, Cunyang received his master's degree from the Institute of Computing Technology (ICT) in 2023. His master's research focused on irregular matrix multiplication, resulting in first-author publications in TPDS, ICPP, and HPCC. His research interests include high performance computing, parallel algorithms, and optimizing AI workloads for modern hardware.