Experiences using cloud computing for a scientific workflow application
Abstract
Clouds are rapidly becoming an important platform for scientific applications. In this paper we describe our experiences running a scientific workflow application in the cloud. The application was developed to process astronomy data released by the Kepler project, a NASA mission to search for Earth-like planets orbiting other stars. This workflow was deployed across multiple clouds using the Pegasus Workflow Management System. The clouds used include several sites within the FutureGrid, NERSC's Magellan cloud, and Amazon EC2. We describe how the application was deployed, evaluate its performance executing in different clouds (based on Nimbus, Eucalyptus, and EC2), and discuss the challenges of deploying and executing workflows in a cloud environment. We also demonstrate how Pegasus was able to support sky computing by executing a single workflow across multiple cloud infrastructures simultaneously.
Additional Information
© 2011 ACM. This material is based upon work supported in part by the National Science Foundation under Grant No. 0910812 to Indiana University for "FutureGrid: An Experimental, High-Performance Grid testbed." Partners in the FutureGrid project include U. Chicago, U. Florida, San Diego Supercomputer Center - UC San Diego, U. Southern California, U. Texas at Austin, U. Tennessee at Knoxville, U. of Virginia, Purdue I., and T.-U. Dresden. G. B. Berriman is supported by the NASA Exoplanet Science Institute at the Infrared Processing and Analysis Center, operated by the California Institute of Technology in coordination with the Jet Propulsion Laboratory (JPL).Additional details
- Eprint ID
- 31560
- DOI
- 10.1145/1996109.1996114
- Resolver ID
- CaltechAUTHORS:20120521-091730036
- 0910812
- NSF
- NASA Exoplanet Science Institute
- Created
-
2012-05-21Created from EPrint's datestamp field
- Updated
-
2021-11-09Created from EPrint's last_modified field
- Caltech groups
- Infrared Processing and Analysis Center (IPAC)