Commit Graph

120 Commits

Author SHA1 Message Date
David S. Miller
47f2c3604f [SPARC64]: Fix X server hangs due to large pages.
This problem was introduced by changeset
14778d9072

Unlike the hugetlb code paths, the normal fault code is not setup to
propagate PTE changes for large page sizes correctly like the ones we
make for I/O mappings in io_remap_pfn_range().

It is absolutely necessary to update all sub-ptes of a largepage
mapping on a fault.  Adding special handling for this would add
considerably complexity to tlb_batch_add().  So let's just side-step
the issue and forcefully dirty any writable PTEs created by
io_remap_pfn_range().

The only other real option would be to disable to large PTE code of
io_remap_pfn_range() and we really don't want to do that.

Much thanks to Mikael Pettersson for tracking down this problem and
testing debug patches.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-08-29 21:23:31 -07:00
David S. Miller
eb398d1044 [SPARC64]: Explicitly print return PC when the kernel fault PC is bogus.
That way we'll have at least some debugging info even if
the stack dump explodes.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-07-24 22:33:58 -07:00
Jörn Engel
6ab3d5624e Remove obsolete #include <linux/config.h>
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
2006-06-30 19:25:36 +02:00
Randy Dunlap
c9cf55285e [PATCH] add poison.h and patch primary users
Localize poison values into one header file for better documentation and
easier/quicker debugging and so that the same values won't be used for
multiple purposes.

Use these constants in core arch., mm, driver, and fs code.

Signed-off-by: Randy Dunlap <rdunlap@xenotime.net>
Acked-by: Matt Mackall <mpm@selenic.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andi Kleen <ak@muc.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-06-27 17:32:38 -07:00
Linus Torvalds
8871e73fdb Merge master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6
* master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6:
  [SPARC]: Add iomap interfaces.
  [OPENPROM]: Rewrite driver to use in-kernel device tree.
  [OPENPROMFS]: Rewrite using in-kernel device tree and seq_file.
  [SPARC]: Add unique device_node IDs and a ".node" property.
  [SPARC]: Add of_set_property() interface.
  [SPARC64]: Export auxio_register to modules.
  [SPARC64]: Add missing interfaces to dma-mapping.h
  [SPARC64]: Export _PAGE_IE to modules.
  [SPARC64]: Allow floppy driver to build modular.
  [SPARC]: Export x_bus_type to modules.
  [RIOWATCHDOG]: Fix the build.
  [CPWATCHDOG]: Fix the build.
  [PARPORT] sunbpp: Fix typo.
  [MTD] sun_uflash: Port to new EBUS device layer.
2006-06-26 10:08:32 -07:00
Anil S Keshavamurthy
d98f8f0518 [PATCH] Notify page fault call chain for sparc64
Overloading of page fault notification with the notify_die() has performance
issues(since the only interested components for page fault is kprobes and/or
kdb) and hence this patch introduces the new notifier call chain exclusively
for page fault notifications their by avoiding notifying unnecessary
components in the do_page_fault() code path.

Signed-off-by: Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-06-26 09:58:22 -07:00
David S. Miller
987c74fcb7 [SPARC64]: Export _PAGE_IE to modules.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-06-25 23:15:03 -07:00
David S. Miller
c2a5a46be4 [SPARC64]: Fix for Niagara memory corruption.
On some sun4v systems, after netboot the ethernet controller and it's
DMA mappings can be left active.  The net result is that the kernel
can end up using memory the ethernet controller will continue to DMA
into, resulting in corruption.

To deal with this, we are more careful about importing IOMMU
translations which OBP has left in the IO-TLB.  If the mapping maps
into an area the firmware claimed was free and available memory for
the kernel to use, we demap instead of import that IOMMU entry.

This is going to cause the network chip to take a PCI master abort on
the next DMA it attempts, if it has been left going like this.  All
tests show that this is handled properly by the PCI layer and the e1000
drivers.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-06-23 23:15:21 -07:00
David S. Miller
486ad10a7e [SPARC64]: Minor bug fix to obp_read_memory().
If we end up zero'ing out the size of one of the entries,
pop it out of the array completely because some code that
examines these things cannot handle a zero length element
properly.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-06-23 23:15:19 -07:00
David S. Miller
372b07bb5a [SPARC64]: Import OBP device tree into kernel data structures.
The basic framework is based on the PowerPC OF code.

This code even tries to get the device addressing components
correct in the full path names.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-06-23 23:15:02 -07:00
David S. Miller
c9f2946fbe [SPARC64]: Disable preemption during flush_tlb_pending().
A context switch will force a call to flush_tlb_pending() (via
switch_to()), so if we test tlb_nr to be non-zero, then sleep, it
would become zero and later back at the original context we'll pass
zero down into the TLB flushing code which should never see a nr
argument of zero.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-04-30 22:54:27 -07:00
David S. Miller
9df1dab1df [SPARC64]: Align address in huge_pte_alloc().
We are about to fill in all HPAGE_SIZE's worth
of PAGE_SIZE ptes, so we have to give the first
pte in that set else we scribble over random memory
when we fill in the ptes.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-31 23:03:36 -08:00
David S. Miller
73c50a27a4 [SPARC64]: Document the instruction checks we do in do_sparc64_fault().
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-31 23:03:35 -08:00
David S. Miller
7bebd83dbf [SPARC64]: Fix off-by-1 error in TSB grow check.
The worst part about this bug is what it would cause
a hugepage TSB to be allocated for every address space
since "0 >= 0".

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-27 01:07:55 -08:00
Nick Piggin
fcab1e5179 [PATCH] sparc64: fix set_page_count merge clash
Merge clash will have broken sparc64. Synch up its online_page
implementation with powerpc, which was identical until the
set_page_count removal.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-03-23 07:15:21 -08:00
Linus Torvalds
d04ef3a795 Merge master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6
* master.kernel.org:/pub/scm/linux/kernel/git/davem/sparc-2.6:
  [SPARC64]: Add a secondary TSB for hugepage mappings.
  [SPARC]: Respect vm_page_prot in io_remap_page_range().
2006-03-22 10:56:57 -08:00
David Gibson
42b88befd6 [PATCH] hugepage: is_aligned_hugepage_range() cleanup
Quite a long time back, prepare_hugepage_range() replaced
is_aligned_hugepage_range() as the callback from mm/mmap.c to arch code to
verify if an address range is suitable for a hugepage mapping.
is_aligned_hugepage_range() stuck around, but only to implement
prepare_hugepage_range() on archs which didn't implement their own.

Most archs (everything except ia64 and powerpc) used the same
implementation of is_aligned_hugepage_range().  On powerpc, which
implements its own prepare_hugepage_range(), the custom version was never
used.

In addition, "is_aligned_hugepage_range()" was a bad name, because it
suggests it returns true iff the given range is a good hugepage range,
whereas in fact it returns 0-or-error (so the sense is reversed).

This patch cleans up by abolishing is_aligned_hugepage_range().  Instead
prepare_hugepage_range() is defined directly.  Most archs use the default
version, which simply checks the given region is aligned to the size of a
hugepage.  ia64 and powerpc define custom versions.  The ia64 one simply
checks that the range is in the correct address space region in addition to
being suitably aligned.  The powerpc version (just as previously) checks
for suitable addresses, and if necessary performs low-level MMU frobbing to
set up new areas for use by hugepages.

No libhugetlbfs testsuite regressions on ppc64 (POWER5 LPAR).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Zhang Yanmin <yanmin.zhang@intel.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: William Lee Irwin III <wli@holomorphy.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-03-22 07:54:04 -08:00
Nick Piggin
7835e98b2e [PATCH] remove set_page_count() outside mm/
set_page_count usage outside mm/ is limited to setting the refcount to 1.
Remove set_page_count from outside mm/, and replace those users with
init_page_count() and set_page_refcounted().

This allows more debug checking, and tighter control on how code is allowed
to play around with page->_count.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2006-03-22 07:54:02 -08:00
David S. Miller
dcc1e8dd88 [SPARC64]: Add a secondary TSB for hugepage mappings.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 01:15:14 -08:00
David S. Miller
14778d9072 [SPARC]: Respect vm_page_prot in io_remap_page_range().
Make sure the callers do a pgprot_noncached() on
vma->vm_page_prot.

Pointed out by Hugh Dickens.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-22 01:15:13 -08:00
David S. Miller
f6b83f070e [SPARC64]: Fix 2 bugs in huge page support.
1) huge_pte_offset() did not check the page table hierarchy
   elements as being empty correctly, resulting in an OOPS

2) Need platform specific hugetlb_get_unmapped_area() to handle
   the top-down vs. bottom-up address space allocation strategies.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:17:17 -08:00
David S. Miller
bb8646d834 [SPARC64]: Optimized TSB table initialization.
We only need to write an invalid tag every 16 bytes,
so taking advantage of this can save many instructions
compared to the simple memset() call we make now.

A prefetching implementation is implemented for sun4u
and a block-init store version if implemented for Niagara.

The next trick is to be able to perform an init and
a copy_tsb() in parallel when growing a TSB table.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:16:41 -08:00
David S. Miller
88d7079458 [SPARC64]: Allow CONFIG_MEMORY_HOTPLUG to build.
online_page() is straightforward, and then add a dummy
remove_memory() that returns -EINVAL just like i386.

There is no point in implementing remove_memory() since
__remove_pages() has no implementation either.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:16:40 -08:00
David S. Miller
9b4006dcf6 [SPARC64]: Use SLAB caches for TSB tables.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:16:39 -08:00
David S. Miller
b52439c22c [SPARC64]: Don't kill the page allocator when growing a TSB.
Try only lightly on > 1 order allocations.

If a grow fails, we are under memory pressure, so do not try
to grow the TSB for this address space any more.

If a > 0 order TSB allocation fails on a new fork, retry using
a 0 order allocation.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:16:38 -08:00
David S. Miller
7a1ac52641 [SPARC64]: Fix and re-enable dynamic TSB sizing.
This is good for up to %50 performance improvement of some test cases.
The problem has been the race conditions, and hopefully I've plugged
them all up here.

1) There was a serious race in switch_mm() wrt. lazy TLB
   switching to and from kernel threads.

   We could erroneously skip a tsb_context_switch() and thus
   use a stale TSB across a TSB grow event.

   There is a big comment now in that function describing
   exactly how it can happen.

2) All code paths that do something with the TSB need to be
   guarded with the mm->context.lock spinlock.  This makes
   page table flushing paths properly synchronize with both
   TSB growing and TLB context changes.

3) TSB growing events are moved to the end of successful fault
   processing.  Previously it was in update_mmu_cache() but
   that is deadlock prone.  At the end of do_sparc64_fault()
   we hold no spinlocks that could deadlock the TSB grow
   sequence.  We also have dropped the address space semaphore.

While we're here, add prefetching to the copy_tsb() routine
and put it in assembler into the tsb.S file.  This piece of
code is quite time critical.

There are some small negative side effects to this code which
can be improved upon.  In particular we grab the mm->context.lock
even for the tsb insert done by update_mmu_cache() now and that's
a bit excessive.  We can get rid of that locking, and the same
lock taking in flush_tsb_user(), by disabling PSTATE_IE around
the whole operation including the capturing of the tsb pointer
and tsb_nentries value.  That would work because anyone growing
the TSB won't free up the old TSB until all cpus respond to the
TSB change cross call.

I'm not quite so confident in that optimization to put it in
right now, but eventually we might be able to and the description
is here for reference.

This code seems very solid now.  It passes several parallel GCC
bootstrap builds, and our favorite "nut cruncher" stress test which is
a full "make -j8192" build of a "make allmodconfig" kernel.  That puts
about 256 processes on each cpu's run queue, makes lots of process cpu
migrations occur, causes lots of page table and TLB flushing activity,
incurs many context version number changes, and it swaps the machine
real far out to disk even though there is 16GB of ram on this test
system. :-)

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:16:33 -08:00
David S. Miller
17b0e199a1 [SPARC64]: Fix 32-bit truncation which broke sparsemem.
The page->flags manipulations done by the D-cache dirty
state tracking was broken because the constants were not
marked with "UL" to make them 64-bit, which means we were
clobbering the upper 32-bits of page->flags all the time.

This doesn't jive well with sparsemem which stores the
section and indexing information in the top 32-bits of
page->flags.

This is yet another sparc64 bug which has been with us
forever.

While we're here, tidy up some things in bootmem_init()
and paginig_init():

1) Pass min_low_pfn to init_bootmem_node(), it's identical
   to (phys_base >> PAGE_SHIFT) but we should use consistent
   with the variable names we print in CONFIG_BOOTMEM_DEBUG

2) max_mapnr, although no longer used, was being set
   inaccurately, we shouldn't subtract pfn_base any more.

3) All the games with phys_base in the zones_*[] arrays
   we pass to free_area_init_node() are no longer necessary.

Thanks to Josh Grebe and Fabbione for the bug reports
and testing.  Fix also verified locally on an SB2500
which had a memory layout that triggered the same problem.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:23 -08:00
David S. Miller
d1112018b4 [SPARC64]: Move over to sparsemem.
This has been pending for a long time, and the fact
that we waste a ton of ram on some configurations
kind of pushed things over the edge.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:22 -08:00
David S. Miller
ee29074d3b [SPARC64]: Fix new context version SMP handling.
Don't piggy back the SMP receive signal code to do the
context version change handling.

Instead allocate another fixed PIL number for this
asynchronous cross-call.  We can't use smp_call_function()
because this thing is invoked with interrupts disabled
and a few spinlocks held.

Also, fix smp_call_function_mask() to count "cpus" correctly.
There is no guarentee that the local cpu is in the mask
yet that is exactly what this code was assuming.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:21 -08:00
David S. Miller
a77754b4d0 [SPARC64]: Bulletproof MMU context locking.
1) Always spin_lock_init() in init_context().  The caller essentially
   clears it out, or copies the mm info from the parent.  In both
   cases we need to explicitly initialize the spinlock.

2) Always do explicit IRQ disabling while taking mm->context.lock
   and ctx_alloc_lock.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:20 -08:00
David S. Miller
f7c00338cf [SPARC64]: Fix loop termination in mark_kpte_bitmap()
If we were aligned, but didn't have at least 256MB left
to process, we would loop forever.

Thanks to fabbione for the report and testing the fix.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:19 -08:00
David S. Miller
74ae998772 [SPARC64]: Simplify TSB insert checks.
Don't try to avoid putting non-base page sized entries
into the user TSB.  It actually costs us more to check
this than it helps.

Eventually we'll have a multiple TSB scheme for user
processes.  Once a process starts using larger pages,
we'll allocate and use such a TSB.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:18 -08:00
David S. Miller
7a591cfe4e [SPARC64]: Avoid dcache-dirty page state management on sun4v.
It is totally wasted work, since we have no D-cache aliasing
issues on sun4v.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:06 -08:00
David S. Miller
2a3a5f5ddb [SPARC64]: Bulletproof hypervisor TLB flushing.
Check TLB flush hypervisor calls for errors and report them.

Pass HV_MMU_ALL always for now, we can add back the optimization
to avoid the I-TLB flush later.

Always explicitly page align the virtual address arguments.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:05 -08:00
David S. Miller
77b838fa1e [SPARC64]: destroy_context() needs to disable interrupts.
get_new_mmu_context() can be invoked from interrupt context
now for the new SMP version wrap handling.

So disable interrupt while taking ctx_alloc_lock in destroy_context()
so we don't deadlock.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:01 -08:00
David S. Miller
a0663a79ad [SPARC64]: Fix TLB context allocation with SMT style shared TLBs.
The context allocation scheme we use depends upon there being a 1<-->1
mapping from cpu to physical TLB for correctness.  Chips like Niagara
break this assumption.

So what we do is notify all cpus with a cross call when the context
version number changes, and if necessary this makes them allocate
a valid context for the address space they are running at the time.

Stress tested with make -j1024, make -j2048, and make -j4096 kernel
builds on a 32-strand, 8 core, T2000 with 16GB of ram.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:14:00 -08:00
David S. Miller
b2bef4424c [SPARC64]: Export _PAGE_E and _PAGE_CACHE to modules.
SBUS flash driver needs it.

Noticed by Fabbione.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:58 -08:00
David S. Miller
d7744a0950 [SPARC64]: Create a seperate kernel TSB for 4MB/256MB mappings.
It can map all of the linear kernel mappings with zero TSB hash
conflicts for systems with 16GB or less ram.  In such cases, on
SUN4V, once we load up this TSB the first time with all the
mappings, we never take a linear kernel mapping TLB miss ever
again, the hypervisor handles them all.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:56 -08:00
David S. Miller
9cc3a1ac9a [SPARC64]: Make use of Niagara 256MB PTEs for kernel mappings.
We use a bitmap, one bit for every 256MB of memory.  If the
bit is set we can use a 256MB PTE for linear mappings, else
we have to use a 4MB PTE.

SUN4V support is there, and we can very easily add support
for Panther cpu 256MB PTEs in the future.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:55 -08:00
David S. Miller
0f15952ac8 [SPARC64]: Export a PAGE_SHARED symbol.
For drivers/media/*, noticed by Fabbione.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:36 -08:00
David S. Miller
8b23427441 [SPARC64]: More TLB/TSB handling fixes.
The SUN4V convention with non-shared TSBs is that the context
bit of the TAG is clear.  So we have to choose an "invalid"
bit and initialize new TSBs appropriately.  Otherwise a zero
TAG looks "valid".

Make sure, for the window fixup cases, that we use the right
global registers and that we don't potentially trample on
the live global registers in etrap/rtrap handling (%g2 and
%g6) and that we put the missing virtual address properly
in %g5.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:34 -08:00
David S. Miller
12e126ad22 [SPARC64]: Check for errors in hypervisor_tlb_lock().
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:31 -08:00
David S. Miller
3f19a84e39 [SPARC64]: Set associativity of kernel TSB descriptor correctly.
It should be 1, not 0.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:29 -08:00
David S. Miller
3b3ab2eb9c [SPARC64]: Use phys tsb address in tsb_insert() in SUN4V.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:25 -08:00
David S. Miller
de635d833f [SPARC64]: Fix flush_tsb_user() on SUN4V.
Needs to use physical addressing just like cheetah_plus.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:07 -08:00
David S. Miller
1daef08a12 [SPARC64]: Fix comment typo in __flush_tlb_kernel_range.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:13:06 -08:00
David S. Miller
bf941d6cd6 [SPARC64]: Log faulting vaddr when bogus kernel PC detected.
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:12:38 -08:00
David S. Miller
cf627156c4 [SPARC64]: Use inline patching for critical PTE operations.
This handles the SUN4U vs SUN4V PTE layout differences
with near zero performance cost.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:12:32 -08:00
David S. Miller
ff02e0d26f [SPARC64]: Move PTE field definitions back into asm/pgtable.h
Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:12:31 -08:00
David S. Miller
c4bce90ea2 [SPARC64]: Deal with PTE layout differences in SUN4V.
Yes, you heard it right, they changed the PTE layout for
SUN4V.  Ho hum...

This is the simple and inefficient way to support this.
It'll get optimized, don't worry.

Signed-off-by: David S. Miller <davem@davemloft.net>
2006-03-20 01:12:25 -08:00