2005-04-16 15:20:36 -07:00
|
|
|
/*
|
|
|
|
* x86 SMP booting functions
|
|
|
|
*
|
|
|
|
* (c) 1995 Alan Cox, Building #3 <alan@redhat.com>
|
|
|
|
* (c) 1998, 1999, 2000 Ingo Molnar <mingo@redhat.com>
|
|
|
|
*
|
|
|
|
* Much of the core SMP work is based on previous work by Thomas Radke, to
|
|
|
|
* whom a great many thanks are extended.
|
|
|
|
*
|
|
|
|
* Thanks to Intel for making available several different Pentium,
|
|
|
|
* Pentium Pro and Pentium-II/Xeon MP machines.
|
|
|
|
* Original development of Linux SMP code supported by Caldera.
|
|
|
|
*
|
|
|
|
* This code is released under the GNU General Public License version 2 or
|
|
|
|
* later.
|
|
|
|
*
|
|
|
|
* Fixes
|
|
|
|
* Felix Koop : NR_CPUS used properly
|
|
|
|
* Jose Renau : Handle single CPU case.
|
|
|
|
* Alan Cox : By repeated request 8) - Total BogoMIPS report.
|
|
|
|
* Greg Wright : Fix for kernel stacks panic.
|
|
|
|
* Erich Boleyn : MP v1.4 and additional changes.
|
|
|
|
* Matthias Sattler : Changes for 2.1 kernel map.
|
|
|
|
* Michel Lespinasse : Changes for 2.1 kernel map.
|
|
|
|
* Michael Chastain : Change trampoline.S to gnu as.
|
|
|
|
* Alan Cox : Dumb bug: 'B' step PPro's are fine
|
|
|
|
* Ingo Molnar : Added APIC timers, based on code
|
|
|
|
* from Jose Renau
|
|
|
|
* Ingo Molnar : various cleanups and rewrites
|
|
|
|
* Tigran Aivazian : fixed "0.00 in /proc/uptime on SMP" bug.
|
|
|
|
* Maciej W. Rozycki : Bits for genuine 82489DX APICs
|
|
|
|
* Martin J. Bligh : Added support for multi-quad systems
|
|
|
|
* Dave Jones : Report invalid combinations of Athlon CPUs.
|
|
|
|
* Rusty Russell : Hacked into shape for new "hotplug" boot process. */
|
|
|
|
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/kernel.h>
|
|
|
|
|
|
|
|
#include <linux/mm.h>
|
|
|
|
#include <linux/sched.h>
|
|
|
|
#include <linux/kernel_stat.h>
|
|
|
|
#include <linux/bootmem.h>
|
2005-06-25 14:54:50 -07:00
|
|
|
#include <linux/notifier.h>
|
|
|
|
#include <linux/cpu.h>
|
|
|
|
#include <linux/percpu.h>
|
2007-03-07 09:12:31 -08:00
|
|
|
#include <linux/nmi.h>
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
#include <linux/delay.h>
|
|
|
|
#include <linux/mc146818rtc.h>
|
|
|
|
#include <asm/tlbflush.h>
|
|
|
|
#include <asm/desc.h>
|
|
|
|
#include <asm/arch_hooks.h>
|
2006-06-26 04:57:01 -07:00
|
|
|
#include <asm/nmi.h>
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
#include <mach_apic.h>
|
|
|
|
#include <mach_wakecpu.h>
|
|
|
|
#include <smpboot_hooks.h>
|
2007-02-13 04:26:21 -08:00
|
|
|
#include <asm/vmi.h>
|
[PATCH] x86: Save the MTRRs of the BSP before booting an AP
Applied fix by Andew Morton:
http://lkml.org/lkml/2007/4/8/88 - Fix `make headers_check'.
AMD and Intel x86 CPU manuals state that it is the responsibility of
system software to initialize and maintain MTRR consistency across
all processors in Multi-Processing Environments.
Quote from page 188 of the AMD64 System Programming manual (Volume 2):
7.6.5 MTRRs in Multi-Processing Environments
"In multi-processing environments, the MTRRs located in all processors must
characterize memory in the same way. Generally, this means that identical
values are written to the MTRRs used by the processors." (short omission here)
"Failure to do so may result in coherency violations or loss of atomicity.
Processor implementations do not check the MTRR settings in other processors
to ensure consistency. It is the responsibility of system software to
initialize and maintain MTRR consistency across all processors."
Current Linux MTRR code already implements the above in the case that the
BIOS does not properly initialize MTRRs on the secondary processors,
but the case where the fixed-range MTRRs of the boot processor are changed
after Linux started to boot, before the initialsation of a secondary
processor, is not handled yet.
In this case, secondary processors are currently initialized by Linux
with MTRRs which the boot processor had very early, when mtrr_bp_init()
did run, but not with the MTRRs which the boot processor uses at the
time when that secondary processors is actually booted,
causing differing MTRR contents on the secondary processors.
Such situation happens on Acer Ferrari 1000 and 5000 notebooks where the
BIOS enables and sets AMD-specific IORR bits in the fixed-range MTRRs
of the boot processor when it transitions the system into ACPI mode.
The SMI handler of the BIOS does this in SMM, entered while Linux ACPI
code runs acpi_enable().
Other occasions where the SMI handler of the BIOS may change bits in
the MTRRs could occur as well. To initialize newly booted secodary
processors with the fixed-range MTRRs which the boot processor uses
at that time, this patch saves the fixed-range MTRRs of the boot
processor before new secondary processors are started. When the
secondary processors run their Linux initialisation code, their
fixed-range MTRRs will be updated with the saved fixed-range MTRRs.
If CONFIG_MTRR is not set, we define mtrr_save_state
as an empty statement because there is nothing to do.
Possible TODOs:
*) CPU-hotplugging outside of SMP suspend/resume is not yet tested
with this patch.
*) If, even in this case, an AP never runs i386/do_boot_cpu or x86_64/cpu_up,
then the calls to mtrr_save_state() could be replaced by calls to
mtrr_save_fixed_ranges(NULL) and mtrr_save_state() would not be
needed.
That would need either verification of the CPU-hotplug code or
at least a test on a >2 CPU machine.
*) The MTRRs of other running processors are not yet checked at this
time but it might be interesting to syncronize the MTTRs of all
processors before booting. That would be an incremental patch,
but of rather low priority since there is no machine known so
far which would require this.
AK: moved prototypes on x86-64 around to fix warnings
Signed-off-by: Bernhard Kaindl <bk@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Andi Kleen <ak@suse.de>
Cc: Andi Kleen <ak@suse.de>
Cc: Dave Jones <davej@codemonkey.org.uk>
2007-05-02 10:27:17 -07:00
|
|
|
#include <asm/mtrr.h>
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-01-30 04:33:11 -08:00
|
|
|
/* which logical CPU number maps to which CPU (physical APIC ID) */
|
2008-03-03 09:12:57 -08:00
|
|
|
u16 x86_cpu_to_apicid_init[NR_CPUS] __initdata =
|
2007-10-19 11:35:03 -07:00
|
|
|
{ [0 ... NR_CPUS-1] = BAD_APICID };
|
2008-01-30 04:33:11 -08:00
|
|
|
void *x86_cpu_to_apicid_early_ptr;
|
2008-03-03 09:12:57 -08:00
|
|
|
DEFINE_PER_CPU(u16, x86_cpu_to_apicid) = BAD_APICID;
|
2007-10-19 11:35:03 -07:00
|
|
|
EXPORT_PER_CPU_SYMBOL(x86_cpu_to_apicid);
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-03-19 10:25:19 -07:00
|
|
|
u16 x86_bios_cpu_apicid_init[NR_CPUS] __initdata
|
|
|
|
= { [0 ... NR_CPUS-1] = BAD_APICID };
|
|
|
|
void *x86_bios_cpu_apicid_early_ptr;
|
|
|
|
DEFINE_PER_CPU(u16, x86_bios_cpu_apicid) = BAD_APICID;
|
|
|
|
EXPORT_PER_CPU_SYMBOL(x86_bios_cpu_apicid);
|
|
|
|
|
2006-09-29 01:58:46 -07:00
|
|
|
u8 apicid_2_node[MAX_APICID];
|
|
|
|
|
2008-03-19 10:25:56 -07:00
|
|
|
extern void map_cpu_to_logical_apicid(void);
|
|
|
|
extern void unmap_cpu_to_logical_apicid(int cpu);
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2005-06-25 14:54:50 -07:00
|
|
|
/* State of each CPU. */
|
|
|
|
DEFINE_PER_CPU(int, cpu_state) = { 0 };
|
|
|
|
|
2008-03-19 10:25:59 -07:00
|
|
|
extern void smp_callin(void);
|
2005-04-16 15:20:36 -07:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Activate a secondary processor.
|
|
|
|
*/
|
2008-03-19 10:25:59 -07:00
|
|
|
void __cpuinit start_secondary(void *unused)
|
2005-04-16 15:20:36 -07:00
|
|
|
{
|
|
|
|
/*
|
2007-05-02 10:27:10 -07:00
|
|
|
* Don't put *anything* before cpu_init(), SMP booting is too
|
|
|
|
* fragile that we want to limit the things done here to the
|
|
|
|
* most necessary things.
|
2005-04-16 15:20:36 -07:00
|
|
|
*/
|
2007-02-13 04:26:21 -08:00
|
|
|
#ifdef CONFIG_VMI
|
|
|
|
vmi_bringup();
|
|
|
|
#endif
|
2007-05-02 10:27:10 -07:00
|
|
|
cpu_init();
|
2005-11-08 21:39:01 -08:00
|
|
|
preempt_disable();
|
2005-04-16 15:20:36 -07:00
|
|
|
smp_callin();
|
2008-03-19 10:25:12 -07:00
|
|
|
|
|
|
|
/* otherwise gcc will move up smp_processor_id before the cpu_init */
|
|
|
|
barrier();
|
2007-02-16 01:27:34 -08:00
|
|
|
/*
|
|
|
|
* Check TSC synchronization with the BP:
|
|
|
|
*/
|
|
|
|
check_tsc_sync_target();
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
if (nmi_watchdog == NMI_IO_APIC) {
|
|
|
|
disable_8259A_irq(0);
|
2008-01-30 04:31:24 -08:00
|
|
|
enable_NMI_through_LVT0();
|
2005-04-16 15:20:36 -07:00
|
|
|
enable_8259A_irq(0);
|
|
|
|
}
|
2005-06-25 14:54:53 -07:00
|
|
|
|
2005-06-25 14:54:54 -07:00
|
|
|
/* This must be done before setting cpu_online_map */
|
|
|
|
set_cpu_sibling_map(raw_smp_processor_id());
|
|
|
|
wmb();
|
|
|
|
|
2005-06-25 14:54:53 -07:00
|
|
|
/*
|
|
|
|
* We need to hold call_lock, so there is no inconsistency
|
|
|
|
* between the time smp_call_function() determines number of
|
2007-10-19 16:13:56 -07:00
|
|
|
* IPI recipients, and the time when the determination is made
|
2005-06-25 14:54:53 -07:00
|
|
|
* for which cpus receive the IPI. Holding this
|
|
|
|
* lock helps us to not include this cpu in a currently in progress
|
|
|
|
* smp_call_function().
|
|
|
|
*/
|
|
|
|
lock_ipi_call_lock();
|
2005-04-16 15:20:36 -07:00
|
|
|
cpu_set(smp_processor_id(), cpu_online_map);
|
2005-06-25 14:54:53 -07:00
|
|
|
unlock_ipi_call_lock();
|
2005-06-25 14:54:56 -07:00
|
|
|
per_cpu(cpu_state, smp_processor_id()) = CPU_ONLINE;
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-03-19 10:25:08 -07:00
|
|
|
setup_secondary_clock();
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
wmb();
|
|
|
|
cpu_idle();
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Everything has been set up for the secondary
|
|
|
|
* CPUs - they just need to reload everything
|
|
|
|
* from the task structure
|
|
|
|
* This function must not return.
|
|
|
|
*/
|
2005-06-25 14:54:55 -07:00
|
|
|
void __devinit initialize_secondary(void)
|
2005-04-16 15:20:36 -07:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* We don't actually need to load the full TSS,
|
2008-01-30 04:30:56 -08:00
|
|
|
* basically just the stack pointer and the ip.
|
2005-04-16 15:20:36 -07:00
|
|
|
*/
|
|
|
|
|
|
|
|
asm volatile(
|
|
|
|
"movl %0,%%esp\n\t"
|
|
|
|
"jmp *%1"
|
|
|
|
:
|
2008-01-30 04:31:02 -08:00
|
|
|
:"m" (current->thread.sp),"m" (current->thread.ip));
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
2005-06-25 14:54:56 -07:00
|
|
|
#ifdef CONFIG_HOTPLUG_CPU
|
|
|
|
void cpu_exit_clear(void)
|
|
|
|
{
|
|
|
|
int cpu = raw_smp_processor_id();
|
|
|
|
|
|
|
|
idle_task_exit();
|
|
|
|
|
|
|
|
cpu_uninit();
|
|
|
|
irq_ctx_exit(cpu);
|
|
|
|
|
|
|
|
cpu_clear(cpu, cpu_callout_map);
|
|
|
|
cpu_clear(cpu, cpu_callin_map);
|
|
|
|
|
|
|
|
unmap_cpu_to_logical_apicid(cpu);
|
|
|
|
}
|
2008-03-19 10:25:43 -07:00
|
|
|
#endif
|
2005-06-25 14:54:56 -07:00
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
static int boot_cpu_logical_apicid;
|
|
|
|
/* Where the IO area was mapped on multiquad, always 0 otherwise */
|
|
|
|
void *xquad_portio;
|
2005-06-23 00:08:33 -07:00
|
|
|
#ifdef CONFIG_X86_NUMAQ
|
|
|
|
EXPORT_SYMBOL(xquad_portio);
|
|
|
|
#endif
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-03-19 10:25:14 -07:00
|
|
|
static void __init disable_smp(void)
|
|
|
|
{
|
2008-03-19 10:25:23 -07:00
|
|
|
cpu_possible_map = cpumask_of_cpu(0);
|
2008-03-19 10:25:25 -07:00
|
|
|
cpu_present_map = cpumask_of_cpu(0);
|
2008-03-19 10:25:14 -07:00
|
|
|
smpboot_clear_io_apic_irqs();
|
|
|
|
phys_cpu_present_map = physid_mask_of_physid(0);
|
|
|
|
map_cpu_to_logical_apicid();
|
|
|
|
cpu_set(0, per_cpu(cpu_sibling_map, 0));
|
|
|
|
cpu_set(0, per_cpu(cpu_core_map, 0));
|
|
|
|
}
|
|
|
|
|
2008-03-19 10:25:13 -07:00
|
|
|
static int __init smp_sanity_check(unsigned max_cpus)
|
2005-04-16 15:20:36 -07:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* If we couldn't find an SMP configuration at boot time,
|
|
|
|
* get out of here now!
|
|
|
|
*/
|
|
|
|
if (!smp_found_config && !acpi_lapic) {
|
|
|
|
printk(KERN_NOTICE "SMP motherboard not detected.\n");
|
2008-03-19 10:25:14 -07:00
|
|
|
disable_smp();
|
2005-10-31 19:16:17 -08:00
|
|
|
if (APIC_init_uniprocessor())
|
|
|
|
printk(KERN_NOTICE "Local APIC not detected."
|
|
|
|
" Using dummy APIC emulation.\n");
|
2008-03-19 10:25:13 -07:00
|
|
|
return -1;
|
2005-10-31 19:16:17 -08:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Should not be necessary because the MP table should list the boot
|
|
|
|
* CPU too, but we do it for the sake of robustness anyway.
|
|
|
|
* Makes no sense to do this check in clustered apic mode, so skip it
|
|
|
|
*/
|
|
|
|
if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
|
|
|
|
printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
|
|
|
|
boot_cpu_physical_apicid);
|
|
|
|
physid_set(hard_smp_processor_id(), phys_cpu_present_map);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If we couldn't find a local APIC, then get out of here now!
|
|
|
|
*/
|
|
|
|
if (APIC_INTEGRATED(apic_version[boot_cpu_physical_apicid]) && !cpu_has_apic) {
|
|
|
|
printk(KERN_ERR "BIOS bug, local APIC #%d not detected!...\n",
|
|
|
|
boot_cpu_physical_apicid);
|
|
|
|
printk(KERN_ERR "... forcing use of dummy APIC emulation. (tell your hw vendor)\n");
|
2008-03-19 10:25:13 -07:00
|
|
|
return -1;
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
2005-10-31 19:16:17 -08:00
|
|
|
verify_local_APIC();
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
/*
|
|
|
|
* If SMP should be disabled, then really disable it!
|
|
|
|
*/
|
2005-10-31 19:16:17 -08:00
|
|
|
if (!max_cpus) {
|
|
|
|
smp_found_config = 0;
|
|
|
|
printk(KERN_INFO "SMP mode deactivated, forcing use of dummy APIC emulation.\n");
|
2007-10-17 09:04:34 -07:00
|
|
|
|
|
|
|
if (nmi_watchdog == NMI_LOCAL_APIC) {
|
|
|
|
printk(KERN_INFO "activating minimal APIC for NMI watchdog use.\n");
|
|
|
|
connect_bsp_APIC();
|
|
|
|
setup_local_APIC();
|
2008-03-19 10:25:49 -07:00
|
|
|
end_local_APIC_setup();
|
2007-10-17 09:04:34 -07:00
|
|
|
}
|
2008-03-19 10:25:13 -07:00
|
|
|
return -1;
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
2008-03-19 10:25:13 -07:00
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Cycle through the processors sending APIC IPIs to boot each.
|
|
|
|
*/
|
|
|
|
static void __init smp_boot_cpus(unsigned int max_cpus)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Setup boot CPU information
|
|
|
|
*/
|
|
|
|
smp_store_cpu_info(0); /* Final full version of the data */
|
|
|
|
printk(KERN_INFO "CPU%d: ", 0);
|
|
|
|
print_cpu_info(&cpu_data(0));
|
|
|
|
|
|
|
|
boot_cpu_physical_apicid = GET_APIC_ID(apic_read(APIC_ID));
|
|
|
|
boot_cpu_logical_apicid = logical_smp_processor_id();
|
|
|
|
|
|
|
|
current_thread_info()->cpu = 0;
|
|
|
|
|
|
|
|
set_cpu_sibling_map(0);
|
2005-04-16 15:20:36 -07:00
|
|
|
|
2008-03-19 10:25:14 -07:00
|
|
|
if (smp_sanity_check(max_cpus) < 0) {
|
|
|
|
printk(KERN_INFO "SMP disabled\n");
|
|
|
|
disable_smp();
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2005-10-31 19:16:17 -08:00
|
|
|
connect_bsp_APIC();
|
|
|
|
setup_local_APIC();
|
2008-03-19 10:25:49 -07:00
|
|
|
end_local_APIC_setup();
|
2005-10-31 19:16:17 -08:00
|
|
|
map_cpu_to_logical_apicid();
|
|
|
|
|
|
|
|
|
2005-04-16 15:20:36 -07:00
|
|
|
setup_portio_remap();
|
|
|
|
|
2005-10-31 19:16:17 -08:00
|
|
|
smpboot_setup_io_apic();
|
|
|
|
|
2007-02-13 04:26:21 -08:00
|
|
|
setup_boot_clock();
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
|
|
|
/* These are wrappers to interface to the new boot process. Someone
|
|
|
|
who understands all this stuff should rewrite it properly. --RR 15/Jul/02 */
|
2007-05-02 10:27:11 -07:00
|
|
|
void __init native_smp_prepare_cpus(unsigned int max_cpus)
|
2005-04-16 15:20:36 -07:00
|
|
|
{
|
2008-03-19 10:25:38 -07:00
|
|
|
nmi_watchdog_default();
|
2005-06-25 14:54:50 -07:00
|
|
|
cpu_callin_map = cpumask_of_cpu(0);
|
|
|
|
mb();
|
2005-04-16 15:20:36 -07:00
|
|
|
smp_boot_cpus(max_cpus);
|
|
|
|
}
|
|
|
|
|
2007-05-02 10:27:11 -07:00
|
|
|
void __init native_smp_prepare_boot_cpu(void)
|
2007-05-02 10:27:10 -07:00
|
|
|
{
|
|
|
|
unsigned int cpu = smp_processor_id();
|
|
|
|
|
2007-05-02 10:27:16 -07:00
|
|
|
init_gdt(cpu);
|
2007-05-02 10:27:10 -07:00
|
|
|
switch_to_new_gdt();
|
|
|
|
|
|
|
|
cpu_set(cpu, cpu_callout_map);
|
|
|
|
__get_cpu_var(cpu_state) = CPU_ONLINE;
|
2005-04-16 15:20:36 -07:00
|
|
|
}
|
|
|
|
|
2008-03-19 10:25:29 -07:00
|
|
|
extern void impress_friends(void);
|
|
|
|
extern void smp_checks(void);
|
|
|
|
|
2007-05-02 10:27:11 -07:00
|
|
|
void __init native_smp_cpus_done(unsigned int max_cpus)
|
2005-04-16 15:20:36 -07:00
|
|
|
{
|
2008-03-19 10:25:29 -07:00
|
|
|
/*
|
|
|
|
* Cleanup possible dangling ends...
|
|
|
|
*/
|
|
|
|
smpboot_restore_warm_reset_vector();
|
|
|
|
|
|
|
|
Dprintk("Boot done.\n");
|
|
|
|
|
|
|
|
impress_friends();
|
|
|
|
smp_checks();
|
2005-04-16 15:20:36 -07:00
|
|
|
#ifdef CONFIG_X86_IO_APIC
|
|
|
|
setup_ioapic_dest();
|
|
|
|
#endif
|
2008-03-19 10:25:37 -07:00
|
|
|
check_nmi_watchdog();
|
2005-04-16 15:20:36 -07:00
|
|
|
zap_low_mappings();
|
|
|
|
}
|