next up previous index
Next: Scalability of Parallel Programming Up: Parallel Programming Previous: Shared Memory Paradigm

Mixing Parallel Programming Paradigms

You can mix various programming paradigms in the same program. For example, you can use Fortran 90 in order to express data parallelism on an SMP node, and then mix it with MPI  library calls in order to make processes running on SMP nodes communicate with each other. If the problem-on-the-node is not data parallel, but can be parallelized by some other means, you can resort to multi-threading on the node and MPI communication between the nodes, or you can multi-thread implicitly by inserting OpenMP directives in your on-the-node program.

Programs written by mixing explicit  multi-threading and MPI are perhaps the most horrible to analyze and write. Yet they are the ones that are going to run most efficiently on clusters of SMPs.

Zdzislaw Meglicki