nasty bug in ppc32 ld.so

[Anton Blanchard]

Hi,

I tracked down why SDET on recent 2.4 and 2.5 kernels sometimes SEGVs.
This happens mostly on RS64, probably due to its large icache.

ld.so tries to optimise away a series of icbis:

      /* Now, we've modified code.  We need to write the changes from
         the data cache to a second-level unified cache, then make
         sure that stale data in the instruction cache is removed.
         (In a multiprocessor system, the effect is more complex.)
         Most of the PLT shouldn't be in the instruction cache, but
         there may be a little overlap at the start and the end.

It dcbst's the entire range but only icbi's the first and last
cachelines. This is clearly invalid, the kernel does not have to
icache synchronise a zero filled page when handing it out to a process,
yet this is exactly what the above behaviour relies on.

Finding this bug was difficult because ld.so did do the dcbst's. So you
could never get an invalid instruction in the icache, just a stale one.
And since it was in the PLT which was always in the same spot, we would
jump through a stale branch in the PLT to the wrong function. Weird
symptoms result :)

I'll submit a patch to the libc guys.

In other news I'll test out enforcing execute permission on 64 bit
binaries, hopefully we can flush out any places we execute out of areas
not marked executable. Lets catch them before it all gets cast in stone.

Anton

--- glibc-2.2.5/sysdeps/powerpc/dl-machine.c~	Sun Sep  9 07:21:34 2001
+++ glibc-2.2.5/sysdeps/powerpc/dl-machine.c	Fri May 10 12:58:04 2002
@@ -311,7 +311,8 @@
 	PPC_DCBST (plt + i);
       PPC_DCBST (plt + size_modified - 1);
       PPC_SYNC;
-      PPC_ICBI (plt);
+      for (i = 0; i < size_modified; i += 4)
+        PPC_ICBI (plt + i);
       PPC_ICBI (plt + size_modified - 1);
       PPC_ISYNC;
     }
@@ -534,5 +535,5 @@
       return;
     }

-  MODIFIED_CODE_NOQUEUE (reloc_addr);
+  MODIFIED_CODE (reloc_addr);
 }

** Sent via the linuxppc64-dev mail list. See http://lists.linuxppc.org/