SC16 Salt Lake City, UT

01. Comparison of High Performance Network Options: EDR InfiniBand vs. 100Gb RDMA Capable Ethernet

Authors: Kari N. Erickson (Los Alamos National Laboratory)Faith V. Van Wig (Los Alamos National Laboratory)Luke A. Kachelmeier (Los Alamos National Laboratory)

Abstract: InfiniBand (IB) has long been the network of choice for high performance computing (HPC). However, advancements in Ethernet and IB technology, as well as other high-performance networks, have made it necessary to analyze the performance of these network options in detail – specifically, we look at 100Gb Ethernet and IB. Advancements in Ethernet include upwards of 100Gb data rates and standardization of RDMA-over-Converged-Ethernet (Routable RoCE). Similarly, IB has introduced Enhanced Data Rate (EDR) hardware, which nearly doubles previous bandwidth, increasing it to 100Gb. The goal of this study is to compare and contrast these two options by looking at their respective bandwidth and latency performance, as well as message injection rates and deployment effort. This research will allow a clear definition of the solution space for the challenges and problems being faced by networks with HPC workloads. LA-UR-16-25359

Poster: pdf
Two-page extended abstract: pdf

Poster Index