[Cbe-oss-dev] GCC inline memcpy() from local store causes bad page fault

[Akinobu Mita]

When I tried to test SPU syscall on PS3, I got Oops:

Unable to handle kernel paging request for unknown fault
Faulting instruction address: 0xd000000000022814
Oops: Kernel access of bad area, sig: 7 [#1]
SMP NR_CPUS=2 
Modules linked in: usbhid hci_usb bluetooth spufs usb_storage ehci_hcd ohci_hcd sg
NIP: D000000000022814 LR: D0000000000227E8 CTR: D0000000000218BC
REGS: c000000002067860 TRAP: 0600   Not tainted  (2.6.21-rc4-git3)
MSR: 8000000000008032 <EE,IR,DR>  CR: 44000488  XER: 20000000
DAR: D0000800803C5F30, DSISR: 0000000040000000
TASK = c0006c0066dec040[2014] 'spe-main' THREAD: c000000002064000 CPU: 0
GPR00: 000000000003FF30 C000000002067AE0 D000000000039138 D000080080386000 
GPR04: D0000800803C5F50 C0000000053AB960 C0000000053AB960 0000000000000000 
GPR08: 0000000000000000 000000000003FFC8 0000000000000000 000000000003FF74 
GPR12: D000000000024088 C000000000402200 00000000100E0000 0000000000000000 
GPR16: 00000000101038D0 00000000100E0000 00000000100A0000 00000000FFC767A0 
GPR20: 000000001001F008 0000000000000000 000000001001F020 0000000000000010 
GPR24: 0000000000000001 C000000002067B60 FFFFFFFFFFFFFFF2 000000000003FF78 
GPR28: C0000000053AB000 D0000800803C5F30 D0000000000382A0 0000000000000000 
NIP [D000000000022814] .spu_process_callback+0xa8/0x1e8 [spufs]
LR [D0000000000227E8] .spu_process_callback+0x7c/0x1e8 [spufs]
Call Trace:
[C000000002067AE0] [2222222222222222] 0x2222222222222222 (unreliable)
[C000000002067BE0] [D000000000022CD4] .spufs_run_spu+0x2a4/0x6c8 [spufs]
[C000000002067CC0] [D00000000001FF7C] .do_spu_run+0xb4/0x180 [spufs]
[C000000002067D80] [C00000000002A654] .sys_spu_run+0xb0/0x108
[C000000002067E30] [C000000000008634] syscall_exit+0x0/0x40
Instruction dump:
4e800421 e8410028 81610070 3d200003 6129ffc8 79600020 3b6b0004 7c03002e 
7f804840 7fa30214 389d0020 419d0118 <7cbd04aa> 7cb905aa 3800ffda 396100a0 


- TRAP: 0600 means alignment interrupt.

- Oops happened at inlined memcpy() in spufs/run.c:spu_process_callback().
  (GCC generated inline copy loop to handle memcpy() instead of kernel
   defined memcpy())

- Instruction dump said alignment interrupt happend by <7cbd04aa>
  ("lswi    r5,r29,32").

  Cell Broadband Engine Programming Handbook said
  an alignment interrupt occurs when
  `The instruction is lmw, stmw, lswi, lswx, stswi, or stswx, and the operand
   is in storage that is caching-inhibited.'
   -- 9.5.8 Alignment Interrupt (x`00..00000600')


I could fix this oops by adding "-fno-builtin-memcpy" to CFLAGS,
or using memcpy_fromio() instead of memcpy() (patch attached).
I don't know what is good fix for this problem. But non privilaged user
can kill SPUs so we need to fix.


Index: 2.6-git-ps3/arch/powerpc/platforms/cell/spufs/run.c
===================================================================
--- 2.6-git-ps3.orig/arch/powerpc/platforms/cell/spufs/run.c
+++ 2.6-git-ps3/arch/powerpc/platforms/cell/spufs/run.c
@@ -260,10 +260,10 @@ int spu_process_callback(struct spu_cont
 	/* get syscall block from local store */
 	npc = ctx->ops->npc_read(ctx);
 	ls = ctx->ops->get_ls(ctx);
-	ls_pointer = *(u32*)(ls + npc);
+	ls_pointer = *(u32 *)(ls + npc);
 	if (ls_pointer > (LS_SIZE - sizeof(s)))
 		return -EFAULT;
-	memcpy(&s, ls + ls_pointer, sizeof (s));
+	memcpy_fromio(&s, ls + ls_pointer, sizeof(s));
 
 	/* do actual syscall without pinning the spu */
 	ret = 0;
@@ -283,7 +283,7 @@ int spu_process_callback(struct spu_cont
 	}
 
 	/* write result, jump over indirect pointer */
-	memcpy(ls + ls_pointer, &spu_ret, sizeof (spu_ret));
+	memcpy_toio(ls + ls_pointer, &spu_ret, sizeof(spu_ret));
 	ctx->ops->npc_write(ctx, npc);
 	ctx->ops->runcntl_write(ctx, SPU_RUNCNTL_RUNNABLE);
 	return ret;