DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling Article Swipe

PDF

Related Concepts

Reinforcement learning Computer science Scheduling (production processes) Distributed computing Supercomputer Hyperparameter Cluster (spacecraft) Computer cluster Parallel computing Artificial intelligence Operating system Mathematical optimization Mathematics

Yuping Fan , Zhiling Lan ·

YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2105.07526 · OA: W3160272839

For decades, system administrators have been striving to design and tune cluster scheduling policies to improve the performance of high performance computing (HPC) systems. However, the increasingly complex HPC systems combined with highly diverse workloads make such manual process challenging, time-consuming, and error-prone. We present a reinforcement learning based HPC scheduling framework named DRAS-CQSim to automatically learn optimal scheduling policy. DRAS-CQSim encapsulates simulation environments, agents, hyperparameter tuning options, and different reinforcement learning algorithms, which allows the system administrators to quickly obtain customized scheduling policies.