Catherine Guelque, Valentin Honoré, Philippe Swartvagher, Gaël Thomas, François Trahay

Identifying performance bottlenecks in a parallel application is tedious, especially because it requires analyzing the behaviour of various software components, as bottlenecks may have several causes and symptoms. For example, a load imbalance may cause long MPI waiting times, or contention on disk may degrade the performance of I/O operations. Detecting a performance problem means investigating the execution of an application and applying several performance analysis techniques. To do so, one can use a tracing tool to collect information describing the behaviour of the application. At the end of the execution, a trace file in a specific format is available to the application user, which can be used to conduct a complete post-mortem investigation. Several challenges emerge from the generation and use of traces. Tracing applications may alter the performance of the application, and can create thousands of heavy trace files, especially at a large scale. Most importantly, the post-mortem analysis needs to load these thousands of trace files in memory, and process them. This quickly becomes impractical for large scale applications, as memory gets exhausted and the number of opened files exceeds the system capacity. In this paper, we propose PALLAS, a generic trace format tailored for conducting various post-mortem performance analysis of traces describing large executions of HPC applications. During the execution of the application, PALLAS collects events and detects their repetitions on-the-fly. When storing the trace to disk, PALLAS groups the data from similar events or groups of events together in order to later speed up trace reading. We demonstrate that the PALLAS online detection of the program structure does not significantly degrade the performance of the applications. Moreover, the PALLAS format allows faster trace analysis compared to other evaluated trace formats. Overall, the PALLAS trace format allows an interactive analysis of a trace that is required when a user investigates a performance problem.

Eyvaz Ahmadzada, Patricia Stolf, Jean-Marc Pierson, Laurent Lefèvre

Computing providers offer flexible, scalable and heterogeneous resources. However, their usage often leads to energy waste due to bad user choices. During a job submission, users may choose a cluster more powerful than needed for workloads, or leave the choice to scheduler. In both cases, the lack of guidance can lead to unnecessary energy consumption. We study this problem by evaluating energy gains achieved by a recommendation system that assists users with more energy efficient cluster choices. We analyze historical workloads from a resource provider to extract consumption and performance patterns. Using these profiles, we recommend clusters that can run similar jobs more efficiently.

To evaluate the accuracy of recommendations, we execute evaluation workloads. Workloads are first executed on a cluster chosen by job scheduler, then submitted to the system to obtain a recommended cluster, and finally executed on the recommended cluster and all other clusters of the computing provider. Results show that the clusters recommended by the system achieve an average relative position score of 0.97, where 0 corresponds to the worst observed cluster for a workload, and 1 to the best

Francieli Boito, Guillaume Pallez, Luan Teylo, Nicolas Vidal

One of the main performance issues faced by high- performance computing platforms is the congestion caused by concurrent I/O from applications. When this happens, the platform’s overall performance and utilization are harmed. From the extensive work in this field, I/O scheduling is the essential solution to this problem. The main drawback of current techniques is the amount of information needed about applications, which compromises their applicability. In this paper, we propose a novel method for I/O management, IO-SETS. We present its potential through a scheduling heuristic called SET-10, which is simple and requires only minimal information. Our extensive experimental campaign shows the importance of IO-SETS and the robustness of SET-10 under various workloads. In particular in most of the simulated scenarios we improve the I/O slowdown over fairshare by 50%, which corresponds in our scenarios to a platform utilization gain of 2.5%. In the practical scenarios that we did, the utilization gain varies between 10 and 30%. We also provide insights on using our proposal in practice.

Lionel Eyraud-Dubois, Théo Grandsart, Philippe Swartvagher

During the last decade, a new kind of computation-intensive and complex application has emerged: Large Language Models (LLM). These applications require the computing power offered by HPC clusters and are complex to implement efficiently: several kinds of parallelisms are possible, limited available memory is often a major constraint, accelerators (GPUs, TPUs, NPUs, ...) need to schedule data transfers and computations, the problem size imposes distributed executions, ... All these challenges are already well-known by developers of the first-class citizen applications running on HPC clusters: linear algebra, numerical simulation, ... That is why task-based runtime systems have been proposed: to ease the writing of HPC applications by providing an abstraction of the machine and its efficient programming. Despite task-based runtime systems being used for a long time now for classic HPC applications, they are not used to implement LLM applications. In this paper, we present our first experiments to try to understand why this is the case and whether using task-based runtime systems for LLM applications (both training and inference) is relevant. We describe our implementation of a small LLM with StarPU, discuss the different choices we had to make and evaluate performance.