Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published 2014 | public
Book Section - Chapter

Towards Managed Terabit/s Scientific Data Flows

Abstract

Scientific collaborations on a global scale, such as the LHC experiments at CERN [1], rely today on the presence of high performance, high availability networks. In this paper we review the developments performed over the last several years on high throughput applications, multilayer software-defined network path provisioning, path selection and load balancing methods, and the integration of these methods with the mainstream data transfer and management applications of CMS [2], one of the major LHC experiments. These developments are folded into a compact system capable of moving data among research sites at the 1 Terabit per second scale. Several aspects that went into the design and target different components of the system are presented, including: evaluation of the 40 and 100Gbps capable hardware on both network and server side, data movement applications, flow management and the network-application interface leveraging advanced network services. We report on comparative results between several multi-path algorithms, the performance increase obtained using this approach, and present results from the related SC'13 demonstration.

Additional Information

© 2014 IEEE. Publication Date 2014-11-16. We thank our commercial partners who made this demonstration possible through donations of the large ensemble of state-of-the art equipment used during SC'13: Mellanox, Brocade, Dell/Force10, Echostreams, Intel, Padtec and Extreme Networks. The development work described in this paper was supported through the following grants: DOE DESC0007346, NSF 0958998, NSF 1246133 and Cisco Research grant 2014-128271. We also thank the SCinet network team for their outstanding support over the past editions of the Supercomputing exhibition, culminating in this year's infrastructure that will reach the Terabit/s scale.

Additional details

Created:
August 19, 2023
Modified:
October 18, 2023