Empirical Studies of Collective Communication in Heterogeneous Cluster Systems

Abstract

All-to-all personalized communication is a very important collective connunication pattern in high performance computing applications. The communication pattern is regular and can be efficiently implemented in a homogenous cluster system. However, due to economical consideration, in today's high-performance cluster computing environments, heterogeneity is very common and processors could have very different computation and communication profile, even in the same cluster. As a result new algorithms must be deployed in order to achieve good communication efficiency in heterogeneous environments. This paper disscus scheduling algorithms for Heterogeneous Networks of Workstations (so called HNOW) systems, and designs simulators to conduct experiments on them. We want to make communication events with all-to-all personalized and all-to-some communication efficiently scheduled, and the final goal is to reduce the completion time of the communication schedule for heterogeneous networks of workstations.