SC16 Salt Lake City, UT

52. Cerberus: A 3-Phase Burst Buffer Aware Batch Scheduler for HPC Systems

Authors: Jiaqi Yan (Illinois Institute of Technology)Xu Yang (Illinois Institute of Technology)Dong Jin (Illinois Institute of Technology)Zhiling Lan (Illinois Institute of Technology)

Abstract: Burst buffer drastically improves the performance of computational applications by providing high perceived IO bandwidth. However, traditional batch schedulers have barely embraced the full potential of burst buffer technology in practice. In this poster, we model the execution of scientific applications that generate tens of TB of data on a supercomputer equipped with burst buffer. We characterize the lifetime of generic applications into three phases: stage-in, running, and stage-out. We develop a novel burst-buffer-aware batch scheduler Cerberus to manage resource allocation in different phases. In both stage-in and stage-out phases, Cerberus allocates the burst buffer resources to achieve the maximum data transfer throughput between burst buffer and the external storage system. In the running phase, Cerberus maximizes the predefined objectives of job scheduling.

Poster: pdf
Two-page extended abstract: pdf

Poster Index