Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences
- Creators
- Bordawekar, Rajesh
Abstract
A majority of parallel applications achieve parallelism by partitioning data over multiple processors. Accessing distributed data structures such as arrays from files often requires each processor to make a large number of small noncontiguous data requests. This problem can be addressed by replacing small non-contiguous requests by large collective requests. This approach, known as collective I/O, has been found to work extremely well in practice. This paper describes implementation and evaluation of a collective I/O prototype in the Parallel File System (PFS) of the Intel Paragon Operating System (Paragon OS). We evaluate the collective I/O performance using its comparison with the PFS MRECORD and KUNIX I/O modes. It is observed that collective I/O provides significant performance improvement over accesses in the H-UNIX mode. However, in many cases, various implementation overheads cause collective I/O to provide lower performance than the H-RECORD I/O mode.
Additional Information
© 1997 ACM. We would like to thank the CACR staff for both computing and administrative support. We would like to thank David Kotz for reviewing the technical report and for answering numerous questions. We would like to thank Tina Pauna for reviewing earlier manuscripts. Finally, we would like to thank anonymous reviewers for their comments. This work was supported in part by the Scalable I/O Initiative, contract number DABT63-94-C-0049 from the Defense Advanced Research Projects Agency(DARPA), administered by the US Army at Fort Huachuca, and Caltech's Center for Advanced Computing Research. Access to Caltech's Intel Paragon was provided by CACR.Additional details
- Eprint ID
- 69539
- Resolver ID
- CaltechAUTHORS:20160810-111243309
- DABT63-94-C-0049
- Defense Advanced Research Projects Agency (DARPA)
- Caltech's Center for Advanced Computing Research
- Created
-
2016-08-10Created from EPrint's datestamp field
- Updated
-
2021-11-11Created from EPrint's last_modified field