2.5 Limitations of parallel processing and Amdahl's law

There are limitations of parallel processing that you must consider when developing parallel applications.

For example, if your application parallelizes perfectly, executing the application on ten processors makes it run ten times faster. However, applications rarely parallelize perfectly because part of the application is serial. This serial component imposes a limit on the amount of parallelization the application can use.
Amdahl’s law describes the maximum speedup that parallel processing can achieve.
The formula for Amdahl’s law is shown in the following figure where the terms in the equation are:
SFraction of the application that is serial.
PFraction of the application that is parallelizable.
NNumber of processors.
Figure 2-4 Formula for Amdahl’s law
To view this graphic, your browser must support the SVG format. Either install a browser with native support, or install an appropriate plugin such as Adobe SVG Viewer.

The following figure shows the speedup that different numbers of processors provide for applications with different serial components.
Figure 2-5 Speedup for an application with different serial components
To view this graphic, your browser must support the SVG format. Either install a browser with native support, or install an appropriate plugin such as Adobe SVG Viewer.

The biggest speedups are achieved with relatively small numbers of processors. However, as the number of processors rises, the per-processor gains are reduced.
You cannot avoid Amdahl’s law in your application but you can reduce the impact.
For high performance with a large number of processors, the application must have a very small serial component. These sorts of applications are said to be embarrassingly parallel.
Related concepts
2.4 Embarrassingly parallel applications
9.5 Reducing the effect of serial computations
Non-ConfidentialPDF file icon PDF versionARM 100614_0300_00_en
Copyright © 2012, 2013, 2015, 2016 ARM. All rights reserved.