Heuritic search of Optimal Reduction Schedule in Heterogeneous Cluster Environments

Abstract

Network of workstation (NOW) is a cost-effective alternative to massively parallel super-computers. As commercially available off-the-shelf processors become cheaper and faster, it is now possible to build a PC or workstation cluster that provides high computing power within a limited budget. However, a cluster may consist of different types of processors and this heterogeneity within a cluster complicates the design of efficient collective communication protocols, and it is a very hard combinatorial problem to design an optimal reduction protocol. Nevertheless, we show that a simple exchange lemma can greatly reduce seraph space so that we can find the optimal solution efficiently. Combined with standard branch-and-nound technique the search space is reduced to be less than one percent of original size for most practical cluster size.