Sun 18 Jun 2023 16:00 - 16:30 at Magnolia 6 - ARRAY: Session 4 Chair(s): Martin Elsman

Today, multiGPU nodes are widely used in high-performance computing and data centers. However, current programming models do not provide simple, transparent, and portable support for automatically targeting multiple GPUs within a node on application areas of array programming. In this paper, we describe a new application programming interface based on the Kokkos programming model to enable array computation on multiple GPUs in a transparent and portable way across both NVIDIA and AMD GPUs. We implement different variations of this technique to accommodate the exchange of stencils (array boundaries) among different GPU memory spaces, and we provide autotuning to select the proper number of GPUs, depending on the computational cost of the operations to be computed on arrays, that is completely transparent to the programmer. We evaluate our multiGPU extension on Summit (#5 TOP500), with six NVIDIA V100 Volta GPUs per node, and Crusher that contains identical hardware/software as Frontier (#1 TOP500), with four AMD MI250X GPUs, each with 2 Graphics Compute Dies (GCDs)for a total of 8 GCDs per node. We also compare the performance of this solution against the use of MPI + Kokkos, which is the cur-rent de facto solution for multiple GPUs in Kokkos. Our evaluation shows that the new Kokkos solution provides good scalability for many GPUs and a faster and simpler solution (from a programming productivity perspective) than MPI + Kokkos.

Sun 18 Jun

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 17:50
ARRAY: Session 4ARRAY at Magnolia 6
Chair(s): Martin Elsman University of Copenhagen, Denmark

#array-sun-magnolia6 Discord icon small YouTube icon small

16:00
30m
Talk
A MultiGPU Performance-Portable Solution for Array Programming Based on Kokkos
ARRAY
Pedro Valero-Lara Oak Ridge National Laboratory, Jeffrey Vetter Oak Ridge National Laboratory
DOI
16:30
30m
Talk
Opportunities for Linear Algebraic Graph Databases
ARRAY
Yuttapichai Kerdcharoen Carnegie Mellon University, Upasana Sridhar Carnegie Mellon University, Tze Meng Low Carnegie Mellon University
17:00
30m
Talk
Towards Structured Algebraic Programming
ARRAY
Denis Jelovina Computing Systems Lab Huawei Zurich Research Center, Daniele Giuseppe Spampinato Computing Systems Lab Huawei Zurich Research Center, Jiawei Zhuang Huawei Technologies Co. Ltd., Albert-Jan Nicholas Yzelman Computing Systems Lab Huawei Zurich Research Center
DOI