[Cbe-oss-dev] Hard crash on fb write past 17MB

[Geert Uytterhoeven]

Here's a patch for 2.6.16.

A fix for 2.6.20 will be released through Geoff's GIT repositories.

Subject: [PATCH] ps3fb: temporary fix for out-of-bounds frame buffer access

From: Geert Uytterhoeven <Geert.Uytterhoeven at sonycom.com>

For both fb_info.screen_base and ps3fb_mmap(), an offset
(FB_OFF(i) + VP_OFF(i)) is added to the start address of the virtual frame
buffer.  This offset is always a non-zero multiple of 64 kiB.

As both fb_write() and ps3fb_mmap() assume a buffer limit based on the actual
frame buffer size, and don't take into account the additional offset, they
allow to write a few multiples of 64 kiB beyond the end of the 18 MiB buffer.
This will overwrite innocent kernel memory, and may cause a crash.

As a temporary fix, this patch prevents out-of-bounds accesses by subtracting
the offset from the exported frame buffer size.

The right solution is to get rid of the offsets completely: as the frame buffer
is virtual, there's no reason it doesn't simply start at the beginning of the
block of allocated memory.

Signed-off-by: Geert Uytterhoeven <Geert.Uytterhoeven at sonycom.com>

---
 drivers/video/ps3fb.c |   17 +++++++++++------
 1 files changed, 11 insertions(+), 6 deletions(-)

--- ps3-linux-src-2.6.16.orig/drivers/video/ps3fb.c
+++ ps3-linux-src-2.6.16/drivers/video/ps3fb.c
@@ -500,6 +500,8 @@ static int ps3fb_check_var(struct fb_var
 static int ps3fb_set_par(struct fb_info *info)
 {
 	int i;
+	unsigned long offset;
+
 	DPRINTK("xres:%d xv:%d yres:%d yv:%d clock:%d\n", 
 		info->var.xres, info->var.xres_virtual, 
 		info->var.yres, info->var.yres_virtual, info->var.pixclock);
@@ -508,8 +510,10 @@ static int ps3fb_set_par(struct fb_info 
 						info->var.bits_per_pixel);
 	i = get_res_table(info->var.xres, info->var.yres);
 	ps3fb.res_index = i;
-	info->screen_base = (char __iomem *)ps3fb.xdr_ea + FB_OFF(i) + VP_OFF(i);
-	memset((void *)ps3fb.xdr_ea, 0, ps3fb.videomemorysize);
+	offset = FB_OFF(i) + VP_OFF(i);
+	info->fix.smem_len = ps3fb.videomemorysize-offset;
+	info->screen_base = (char __iomem *)ps3fb.xdr_ea + offset;
+	memset(ps3fb.xdr_ea, 0, ps3fb.videomemorysize);
 	return 0;
 }
 
@@ -644,7 +648,7 @@ static int ps3fb_mmap(struct fb_info *in
 	i = get_res_table(info->var.xres, info->var.yres);
 	if (i == -1) return -EINVAL;
 	
-	if (offset + size > ps3fb.videomemorysize)
+	if (offset + size > info->fix.smem_len)
 		return -EINVAL;
 	offset += ps3fb.videomemory_phys + FB_OFF(i) + VP_OFF(i);
 	if (remap_pfn_range(vma, vma->vm_start, offset >> PAGE_SHIFT,
@@ -1036,6 +1040,7 @@ static int __init ps3fb_probe(struct pla
 	uint64_t lpar_reports_size =0;
 	uint64_t xdr_lpar;
 	uint64_t status;
+	unsigned long offset;
 
 	/* get gpu context handle */
 	status = lv1_gpu_memory_allocate(
@@ -1089,13 +1094,13 @@ static int __init ps3fb_probe(struct pla
 
 	ps3fb_fb_var_default();
 
-	info->screen_base = (char __iomem *)ps3fb.xdr_ea + 
-			FB_OFF(ps3fb.res_index) + VP_OFF(ps3fb.res_index);
+	offset = FB_OFF(ps3fb.res_index) + VP_OFF(ps3fb.res_index);
+	info->screen_base = (char __iomem *)ps3fb.xdr_ea + offset;
 	info->fbops = &ps3fb_ops;
 
 	info->var = ps3fb_default;
 	info->fix = ps3fb_fix;
-	info->fix.smem_len = ps3fb.videomemorysize;
+	info->fix.smem_len = ps3fb.videomemorysize-offset;
 	info->pseudo_palette = info->par;
 	info->par = NULL;
 	info->flags = FBINFO_FLAG_DEFAULT;

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- Sony Network and Software Technology Center Europe (NSCE)
Geert.Uytterhoeven at sonycom.com ------- The Corporate Village, Da Vincilaan 7-D1
Voice +32-2-7008453 Fax +32-2-7008622 ---------------- B-1935 Zaventem, Belgium