Data Alignment for Parallelizing Compiler on Distributed-Memory Multiprocessors

Abstract

In distributed-memory multiprocessors system, the distribution of data across processors is critically important to the efficiency in executing a data parallel program. This is due to how this distribution is done will affect the amount of data movement required and the interprocessor communication is much more expensive than computation. In this paper, we analyze the reference patterns in a program where multiple arrays used have affinity relations. The abstract machine we adopt is a D-dimensional grid of N1×N2×…×ND processors, where D is the smallest dimensionality of arrays used in the program. Based on the reference patterns, we present a scheme to find a maximum spanning tree in the alignment group. For each reference pattern in the maximum spanning for these two dimensions of arrays to the same place of template, such that the reference pattern with maximum cost will get free communication in all distribution fashions and in any number of processors.