There are limitations of parallel processing that you must consider when developing parallel applications.
For example, if your application parallelizes perfectly, executing the
application on ten processors makes it run ten times faster. However, applications rarely
parallelize perfectly because part of the application is serial. This serial component imposes
a limit on the amount of parallelization the application can use.
Amdahl’s law describes the maximum speedup that parallel processing can
The formula for Amdahl’s law is shown in the following figure where the terms
in the equation are:
|S||Fraction of the application that is serial.
|P||Fraction of the application that is parallelizable.
|N||Number of processors.
Figure 2-4 Formula for Amdahl’s law
The following figure shows the speedup that different numbers
of processors provide for applications with different serial components.
Figure 2-5 Speedup for an application with different
The biggest speedups are achieved with relatively small numbers of
processors. However, as the number of processors rises, the per-processor gains are
You cannot avoid Amdahl’s law in your application but you
can reduce the impact.
For high performance with a large number of processors, the application must
have a very small serial component. These sorts of applications are said to be embarrassingly parallel.