The roundRobin program was meant only to illustrate basic interprocessor communication, so we did not really focus on data. Processor 0 generated the data. Then each processor received and sent the data back and forth.
But in real-world programming, it's never that simple! In most cases, data will have to be shared among the processors so that each cares about only its own portion of the data. "Data decomposition" refers to the sharing data between parallel processes in the best possible (i.e. most efficient) fashion. No process should step on another process' data either.
More parallelism is good; maximum amount of CPU time must be spent by each processor on computation (and not message passing)
Least interprocessor communication is desirable.
Objective: How can we decompose data in our SPMD model in order to maximize parallelization while minimizing interprocessor communication?
Identify the code segment(s) that may be divided up for simultaneous work by separate processors.
Determine the interprocessor communication necessary for this division of labor.
The parallelizable segment(s) should account for most of the serial code's run-time. The interprocessor communication should be at most occasional.
"Embarrassingly parallelizable" serial code is code in which almost 100% of run time is spent in parallelizable code segments, while the interprocessor communication requirements are relatively insignificant.
In the following section we will consider a problem for which the serial code is embarrassingly parallelizable, a problem of one-dimensional numerical integration.
| Previous: RoundRobin - MPI calls... | Up: Table of Contents | Next: A Naturally Parallelizable Computation |
|---|