SC16 Salt Lake City, UT

88. Attributed Consistent Hashing for Heterogeneous Storage Systems

Authors: Jiang Zhou (Texas Tech University)Wei Xie (Texas Tech University)Yong Chen (Texas Tech University)

Abstract: Cloud-scale storage system is an important building block of the cloud infrastructure. It demands the flexibility to distribute data and provide high I/O performance. Consistent hashing algorithm is widely used in large-scale parallel/distributed storage systems for the decentralized design, scalability and adaptability. It can evenly distribute data among nodes but lacks efficiency in a heterogeneous environment. In this research, we propose a novel data placement algorithm, which is based on consistent hashing while making it more efficient for heterogeneous storage systems. By considering both the capacity and bandwidth attributes of nodes, our algorithm can make better use of heterogeneous devices. Our current proof-of-concept evaluations with a distributed storage system, Sheepdog, show promising results.

Poster: pdf
Two-page extended abstract: pdf

Poster Index