4.7.1. Booting

Initialization of the external system might have to be synchronized among cores. Typically, only one of the cores in the system must run code that initializes the memory system and peripherals. Similarly, the initialization of an SMP operating system typically runs on only one core - the primary core. When the system is fully booted, the remaining cores are brought online and this distinction between the primary core and the secondary cores is lost.

If all of the cores come out of reset at the same time, they normally all start executing from the same reset vector. The boot code then reads the cluster ID to determine which core is the primary. The primary core performs the initialization and then signals to the secondary ones that everything is ready. An alternative method is to hold the secondary cores in reset while the primary core does the initialization. This method requires hardware support to coordinate the reset.

Processor ID

Booting provides a simple example of a situation where particular operations must be performed only on a specific core. Other operations perform different actions, depending on the core on which they are executing.

The CP15:MPIDR Multiprocessor Affinity Register provides an identification mechanism in a multi-core system.

This register was introduced in ARMv7, but was in fact already used in the same format in the ARM11 MPCore. In its basic form, it provides up to three levels of affinity identification, with eight bits identifying individual blocks at each level, that is, Affinity Level 0, 1, and 2.

This information can also be useful to an operating system scheduler, as it indicates the magnitude order of the cost of migrating a process to a different core, processor, or cluster.

The format of the register is slightly extended with the ARMv7-A multiprocessing extensions. This extends the previous format by adding an identification bit to reflect that this is the new register format, and adds the U bit that indicates whether the current core is the only core in a single-core implementation.

SMP boot in Linux

The boot process comprises the following main stages:

  1. The kernel decompresses itself.

  2. The processor-dependent kernel code is executed to initialize the CPU and memory.

  3. The processor-independent kernel code is executed to start the ARM Linux SMP kernel by booting up all cores and initialize all the kernel components and data structures.

Figure 4-6 shows the boot process of the ARM Linux kernel:

Figure 4.6. Booting flowchart for the ARM Linux Kernel

To view this graphic, your browser must support the SVG format. Either install a browser with native support, or install an appropriate plugin such as Adobe SVG Viewer.

To boot the primary core, the kernel allocates a 4KB page as the vector page. It maps this using devicemaps_init() in the file arch/arm/mm/mmu.c to the location of the exception vectors, virtual address 0xFFFF0000 or 0x00000000. This step is invoked very early in the ARM system boot.

When this step is complete, trap_init in arch/arm/kernel/traps.c copies the exception vector table, exception stubs, and kuser helpers into the vector page. You must copy the exception vector table to the start of the vector page. Use a series of memcpy() operations to copy the exception stubs to address 0x200, and copy kuser helpers to the top of the page, at 0x1000 - kuser_sz.

The method for booting the secondary cores can differ somewhat, depending on SoC. The primary core invokes the boot_secondary() method to get a secondary core booted into the operating system. You must implement this method for each mach type that supports SMP. Most of the other SMP boot functionality is extracted out into generic functions in linux/arch/arm/kernel.

The method below describes the process on an ARM Versatile Express development board, mach-vexpress.

While the primary core is booting, the secondary cores are held in a standby state, using the WFI instruction. The primary core provides a startup address for the secondary cores and wakes them using an Inter-Processor Interrupt (IPI), an SGI signaled through the GIC. Booting of the secondary cores is serialized, using the global variable pen_release. Conceptually, you can think of the secondary cores being in a holding pen and being released one at a time, under the control of the primary core. The variable pen_release is set by the kernel code to the ID value of the processor to boot and then reset by that core when it has booted. When an inter-processor interrupt occurs, the secondary core checks the value of pen_release against their own ID value by using the MPIDR register.

Booting of the secondary core proceeds in a way similar to the primary core. The secondary core enables the MMU. It enables the interrupt controller interface to itself and calibrates the local timers. It sets a bit in cpu_online_map and calls cpu_idle(). The primary processor can detect the setting of the appropriate bit in cpu_online_map and set pen_release to the next secondary core.

Copyright © 2014 ARM. All rights reserved.ARM DAI0425