FSL 64-bit DMA window question

[Scott Wood]

On 06/07/2013 05:09:54 PM, Benjamin Herrenschmidt wrote:
> On Fri, 2013-06-07 at 17:02 -0500, Scott Wood wrote:
> > O
> > > I thought the device tree was for describing the hardware, rather
> > > than configuration? :-)
> 
> A bit of both really. See things like /chosen etc...
> 
> Also to what extent a MAC address is HW vs. configuration ? :-)

Normally I'd consider that hardware, unless you're overriding it for  
some reason.

> The HW configuration for a given boards (ie, internal address map,
> location of the various bridge windows etc...) is a fairly common  
> thing
> to put in a device-tree.

Sure, there's some stuff that is technically software config, but which  
is easier to treat as a fixed property of the hardware once U-Boot is  
done setting it up.  But U-Boot doesn't have anything to do with this  
DMA window...

> If the PCI outbound windows are there (and they are), why not the
> inbound ones ? IE, it's not far fetched.

PCI outbound windows (and the LAWs that back them up) are set up by  
U-Boot.  With the current FSL kernel code, if you set it to something  
that isn't covered by a PCIe LAW from U-Boot, it won't work.

> > > A kernel command line option might be more appropriate, unless you
> > > just mean describing the difference between e6500 (which supports  
> 40
> > > bit addresses) and previous chips (which support 36 bits), rather
> > > than an ability to move it earlier even on e6500.
> 
> Well, so we could indeed locate it at 36 on e5500 and that would clear
> the current use case and break again on e6500... or we can make it
> depend on the overall address map of the board, which is described
> in the device-tree. IE, if you don't "locate" anything above 39-bit
> in our address map, then using 39 for the window is ok. IE. It's a
> choice. A server board setup that doesn't need gfx but want address
> space for some other things wouldn't care.

I suppose we could put in the device tree the highest address that has  
been configured for anything...  though for that to be useful we'd need  
to get out of the habit of putting CCSR near the top of the physical  
address space.

> > > Maybe we could by default use the size of actual RAM, rather than  
> the
> > > physical address space.  Then only odd scenarios such as DMA to
> > > non-kernel-owned RAM would need manual adjustment (MSIs would  
> still
> > > go through the special window below 4G).
> 
> You also need to account for other on-chip MMIOs no ? Or do you never
> intend to let PCI devices hit them ?

I don't think we normally need that (other than MSIs, which have a  
special window under 4G)...  The question is whether it's better to let  
odd use cases work without having to manually move the DMA window, at  
the cost of decreased out-of-the-box performance with common PCIe cards.

-Scott