Introduction

Supercomputers have revolutionized computational science by enabling high-performance computing at an unprecedented scale. Traditionally, these systems have relied on double-precision arithmetic for accuracy in simulations. However, with the advent of GPUs and their ability to perform mixed-precision calculations at faster speeds, there is a growing need for benchmarking tools that can evaluate the performance of supercomputers in this context. In response to this demand, researchers at the Oak Ridge National Laboratory (ORNL) have developed OpenMxP, an open-source benchmarking software package specifically designed for testing lower-precision performance on GPU-accelerated supercomputers.

The Need for Mixed-Precision Benchmarking

Supercomputers, such as the Frontier system at ORNL, are at the forefront of computational science, pushing the boundaries of what is possible in terms of computing power. However, accurately measuring the performance of these systems on mixed-precision calculations has been a challenge. While GPUs are capable of performing calculations with 16 or 32 bits of precision, there has been a lack of widely available, open-source software for testing the performance of these lower-precision calculations at extreme scale. This gap in understanding prompted the development of OpenMP.

Feiyi Wang, the leader of the Analytics and AI Methods at Scale (AAIMS) group at the National Center for Computational Sciences at ORNL, emphasized the importance of benchmarking in improving supercomputing systems. He stated, “You cannot improve it if you cannot measure it, which highlights the importance of benchmarking. This reference implementation of OpenMP as a capability benchmark will benefit all the other leadership computing systems.”

Introducing OpenMP

OpenMxP is a cross-platform benchmarking software package that implements the HPL-MxP benchmarking task. Introduced in 2019, HPL-MxP is the industry standard for measuring the mixed-precision performance of supercomputing systems. Unlike previous benchmarking tools, HPL-MxP presents a problem to solve—a dense system of linear equations—leaving the software to solve it up to the benchmarkers. This approach allows for flexibility and customization in evaluating the performance of different supercomputer systems.

One of the key challenges in developing OpenMP was the need to create benchmarking codes specifically tailored to the next-generation AMD CPUs and GPUs used in the Frontier system. Previously, proprietary codes developed by GPU chip vendors were used to evaluate the speed of mixed-precision calculations on supercomputers. However, with Frontier’s unique hardware configuration, new benchmarking codes had to be developed. This led to the formation of a team at ORNL, which began studying the benchmark problem and consulting scientists with experience in similar areas.

Overcoming Challenges

Developing OpenMP was not without its challenges. The team faced unexpected issues with software stacks and had to navigate a learning curve to ensure the code worked as intended. Mike Matheson, the technical lead of the OpenMP project, highlighted the iterative process they went through, saying, “We would try things, and then it wouldn’t work, and then we’d talk to other people, and then we’d try something else. We were kind of just probing ahead, trying to figure out what actually worked.”

To test OpenMP before the completion of the Frontier system, the team leveraged the Summit supercomputer, which was already available. This allowed them to fine-tune the code and ensure its scalability. Once OpenMxP was ready, it was deployed on Frontier, and the results were impressive. The initial mixed-precision benchmark of 6.86 exaflops (6.86 billion billion floating-point operations per second) put Frontier at the top of the 2022 HPL-MxP list. One year later, Frontier achieved a staggering 9.95 exaflops, further solidifying its position as a computational powerhouse.

The Power of OpenMP

While OpenMxP’s ability to provide numerical results for benchmarking purposes is noteworthy, its true strength lies in the insights it provides for optimizing the performance of GPU/CPU supercomputers. By revealing how small changes in programming can lead to significant improvements in computational speed, OpenMxP helps researchers fine-tune their code and make the most of the powerful hardware available to them.

The emergence of GPUs capable of performing low-precision calculations faster than CPUs has opened up new possibilities in scientific simulations. Traditionally, simulations relied on double-precision calculations, but with the advent of specialized hardware, there is a growing interest in exploring the advantages of mixed-precision calculations. OpenMxP serves as a tool to demonstrate these advantages to computational scientists and encourages them to embrace GPU-equipped systems capable of performing mixed-precision calculations.

Furthermore, OpenMxP is not limited to benchmarking alone. It can be used as a solver for various scientific and engineering problems, enabling researchers to tackle large-scale computations in a faster and more energy-efficient manner. For example, the ORNL team used OpenMxP as a solver for TwoFold, a software stack that predicts how drug molecules interact with pathogens and determines the 3D structure of the interaction. The application of OpenMxP in this context highlights its potential as a tool for solving complex problems in science and engineering.

Conclusion

OpenMxP represents a significant advancement in the field of benchmarking for supercomputers. By providing an open-source, cross-platform benchmarking software package, researchers at ORNL have addressed the need for evaluating the performance of lower-precision calculations on GPU-accelerated supercomputers. OpenMxP not only allows for accurate benchmarking but also offers insights into optimizing the performance of supercomputers and encourages the adoption of mixed-precision calculations in scientific simulations. With its ability to serve as a solver for various problems, OpenMxP opens up new possibilities for scientific and engineering research at an unprecedented scale.