HARP: Predictive Transfer Optimization Based on Historical Analysis and Real-Time Probing
SessionTopics in Distributed Computing
Session ChairAlexandru Iosup
Event Type
Paper
Advanced
Intermediate
Networks
TimeTuesday, November 15th3:30pm - 4pm
Location355-E
DescriptionIncreasingly data-intensive applications require frequent movement of large datasets from one site to the other. Despite the growing capacity of the networking capacity, these data movements rarely achieve the promised data transfer rates due to poorly tuned data transfer protocols. In this paper, we present predictive end-to-end data transfer optimization algorithms based on historical data analysis and real-time background traffic probing, dubbed HARP. Most of the existing work in this area is solely based on real time network probing, which either cause too much sampling overhead or fail to accurately predict the correct transfer parameters. Combining historical data analysis with real time sampling enables our algorithms to tune the application level data transfer parameters accurately to achieve close-to-optimal end-to-end data transfer throughput with very low overhead. Our experimental analysis over a variety of network settings shows that HARP outperforms existing solutions by up to 50% in terms of achieved throughput.
