FU Berlin, Fachbereich Mathematik und Informatik, Institut für Informatik

Vortrag des Informatik-Kolloquiums

This talk describes ongoing research in which we are building a Java-based Grid programming environment for distributed supercomputing applications, called Ibis[3]. We use a Java-centric approach, because Java's high portability ("write once, run everywhere") allows applications to run on a heterogeneous grid without any effort for porting or recompilation. A major problem with using Java for grid computing, however, is the inferior performance and limited expressiveness of Remote Method Invocation (RMI). Earlier projects (e.g., Manta[1]) solved the performance problem, but at the cost of using a runtime system written in native code, which gives up Java's high portability. The philosophy behind Ibis is to try to obtain good performance without using any native code, but allow native solutions to further optimize special cases. For example, a Grid application developed with Ibis can use a pure-Java RMI implementation over TCP/IP that will run "everywhere"; if the application runs on, say, a Myrinet cluster, Ibis can load a more efficient RMI implementation for Myrinet that partially uses native code.

The talk will first give a motivation for this work by showing that distributed supercomputing is feasible for a much wider range of applications than one might expect. Parallel processing on grids is of course challenging, given the relatively slow wide-area networks. In particular, dealing with very high latencies is a difficult problem, so most research in this area so far is restricted to "trivially parallel" applications. In our earlier work, however, we have shown that many medium-grained parallel algorithms can be optimized to run efficiently on a hierarchically structured grid, such as a geographically distributed collection of clusters or supercomputers [4]. Examples of such hierarchical grids are the DAS (Distributed ASCI Supercomputer) and DAS-2 systems, which each consist of multiple Myrinet-based cluster computers distributed over The Netherlands.

The second part of the talk will discuss the design, partial implementation, and early performance experiments of Ibis. The Ibis programming environment consists of a communication runtime system with a well-defined interface and a range of communication paradigms implemented on top of this interface, including object replication, group communication, and collective communication, all integrated cleanly into Java. Ibis has also been used to implement Satin[2], which is a Cilk-like wide-area divide-and-conquer system in Java. The pure-Java implementation of Ibis boosts RMI performance by avoiding the high overhead of runtime type inspection that current RMI implementations have. The special-case implementations do more aggressive optimizations, even allowing zero-copy communication in certain cases.

[1] Jason Maassen, Rob van Nieuwpoort, Ronald Veldema, Henri Bal, Thilo Kielmann, Ceriel Jacobs, and Rutger Hofman: "Efficient Java RMI for Parallel Programming", ACM. Trans. on Programming Languages and Systems, Vol. 23, No. 6, pp. 747-775, Nov. 2001.