A Scaleable Scheme for Causal Message Ordering

Abstract

This paper presents a scalable scheme for ensuring causal ordering of messages passing among processes in large-scale distributed systems. Previously proposed approaches, categorized as centralized or fully distributed, either place the entire processing loads on a single process or incur quadratic message overheads in the number of participating processes. These solutions perform limitedly in large-scale systems. Our scheme organizes the entire system as hierarchical clusters in which any of the previously proposed approaches can be employed. Message causality is maintained by enforcing the rules by which messages are propagated from origins to destinations. This approach incurs much less processing load on hot-spot sites than the centralized approach, or, alternatively, requires a message space overhead much less than that in the fully distributed approach. We shall show that, by setting cluster size appropriately, the message space overhead can be only a linear or even a logarithmic function of the number of processes involved. Our approach is suitable for large non-proprietary networks.