[PATCH] [POWERPC] Reintroduce O_SYNC flag to make DRAM non-cached.

[Spence Nick]

Paul Mackerras wrote:
> This says nothing about why you want to do this---what bad 
> things arise from the present code, or what good things would 
> happen if the change was made.  It needs to.

We found the problem when porting code from Linux 2.4 to 2.6, where a
user space application maps a 1 MByte region of DRAM with the O_SYNC 
flag to communicate with an internal core that shares access to the
DRAM but does not have any cache snooping logic.

In 2.4 the mem driver honors the O_SYNC flag and makes the requested
memory
memory non-cached so that writes from user space are immediately
available
to the second core.

In 2.6 the powerpc mem driver no longer honors the O_SYNC flag so the
data
is only written out to DRAM (and becoming visible to the second core) as
the
cache reloads with other data.

> The change isn't acceptable as it is, because IBM machines 
> with a hypervisor don't allow system memory to be mapped 
> uncacheable.  It would at least have to have some 
> platform-specific check in there.
> 
> On other powerpc platforms, this would introduce possible 
> cache paradoxes, where a page is mapped cacheable at one 
> address (via the linear mapping) and uncacheable at another 
> address (via /dev/mem).  If you really really want to do 
> this, you'll need to add code to bust up the large pages (or 
> TLB entries) used for the linear mapping so that the 
> corresponding pages of the linear mapping can be unmapped, or 
> mapped cacheable.  Plus you'll need to add the appropriate 
> cache flushing code to make sure there aren't any cache lines 
> for the page in cache.  Further, you'll need to arrange 
> somehow to make sure that the kernel stack has a resident TLB 
> entry during the unrecoverable parts of the exception entry 
> and exit paths.
> 

This is getting to be a much more elaborate project.

In our case the memory area is reserved so that the main processor will
not
touch it except when mapped by the mem driver.

Alternative solutions are to modify the user space code to flush out the
caches by reading a large area, or to implement a board specific 
ppc_md.phys_mem_access_prot() function to do this.

Either of these will meet my needs, but this will leave the O_SYNC flag
ignored on the PowerPC architecture in 2.6, but honored by Linux 2.4 and
other architectures. Probably not a problem for most people.

Thanks,
  Nick