Battle of the Defaults: Extracting Performance Characteristics of HDF5 under Production Load
Published in CCGrid, 2021
Popular parallel I/O libraries, such as HDF5, pro-vide tuning parameters to obtain superior performance. However,the selection of effective parameters on production systems iscomplex due to the interdependence of I/O software and filesystem layers. Hence, application developers typically use thedefault parameters and often experience poor I/O performance.This work conducts a benchmarking-based analysis on theHDF5 behaviors with a wide variety of I/O patterns to extractperformance characteristics under the production workload. Tomake the analysis well controlled, we exercise I/O benchmarkson POSIX-IO, MPI-IO, and HDF5 using the same I/O patternsand in the same jobs. To address high performance variabilityin production environments, we repeat the benchmarks acrossI/O patterns, storage devices, and time intervals. Based on theresults, we identified consistent HDF5 behaviors that appropriateconfigurations and operations on dataset layout and file-metadataplacement can improve performance significantly. We apply ourfindings and evaluate the tuned I/O library on two supercom-puters: Summit and Cori. The results show that our tunedparameters can achieve more than 10×I/O performance speedupthan that with default parameters on both systems, suggesting the effectiveness, stability, and generality of our solution.