Efficient Parallel I/O Scheduling For Clusters

Abstract

This paper studies scheduling of parallel I/O operations in cluster environments. Due to dynamic nature of clusters, fault tolerance has been a crucial issue in executing data-intensive applications on clusters of workstations/PCs. One key strategy for enhancing fault tolerance is data replication, that is, replicating multiple copies of a data le and striping them over multiple disks. Parallel I/O scheduling aims to reducing nish time of a batch of parallel I/O operations by nding an optimal sequence in executing these I/O oprations. Most existing works in parallel I/O scheduling, however, have not taken data replication into consideration. In this paper, we study parallel I/O scheduling for systems that provide data replication. The main contribution of this paper is a complexity analysis of the scheduling problem and a set of scheduling algorithms. We show that nding an optimal schedule for systems that provide data replication is NP-complete, and then we propose a set of heuristic algorithms. We evaluate the I/O performance of these algorithms using a software simulator and report our experimental results.