74. Meta-Balancer: Automating Load Balancing Decisions
Authors: Harshitha Menon (University of Illinois)Kavitha Chandrasekar (University of Illinois)Laxmikant Kale (University of Illinois)
Abstract: HPC applications are increasingly becoming complex and dynamic. Many applications require dynamic load balancing to achieve high performance. Different applications have different characteristics and hence need to use different load balancing strategies. There are many load balancing algorithms available. However, invocation of an unsuited load balancing strategy can lead to inefficient execution. Most commonly, the application programmer decides which load balancer to use based on some educated guess. We propose Meta-Balancer, a framework to automatically decide the best suited load balancing strategy. Meta-Balancer monitors application characteristics and based on that, it chooses an ideal load balancing algorithm to use. In order to predict the best load balancing strategy, Meta-Balancer uses a supervised random forest machine learning technique with the application characteristics as the features. Using this, we are able to achieve high prediction accuracy of 82% on the test set to demonstrate performance benefits of up to 3X.
Two-page extended abstract: pdf