[RFC] DT affinity bindings/representing bus masters with DT

[Lorenzo Pieralisi]

Thank you for having a look Grant.

On Sat, Apr 13, 2013 at 11:24:48PM +0100, Grant Likely wrote:
> On Mon, 25 Mar 2013 15:20:39 +0000, Lorenzo Pieralisi <lorenzo.pieralisi at arm.com> wrote:
> > On Tue, Mar 19, 2013 at 07:07:34AM +0000, David Gibson wrote:
> > > On Mon, Mar 18, 2013 at 09:48:16AM +0000, Lorenzo Pieralisi wrote:
> > > > > > I think we need to post what we have, it is really complex to explain
> > > > > > the issue without a concrete example. To cut a long story short I
> > > > > > would not say that the resources sit in different address spaces, it is
> > > > > > that we need to associate those address ranges with specific bus masters.
> > > > > > 
> > > > > > We have to have a way to say:
> > > > > > 
> > > > > > "Address range 0x80001000 - 0x80001fff is used to programme the control
> > > > > > registers associated with the port connected to master X".
> > > > > > 
> > > > > > When a CPU wants to programme a control port for a specific master, it
> > > > > > needs to know what address range should be programmed.
> > > > > > 
> > > > > > I mentioned "resources" instead of addresses since the problem we are
> > > > > > having is the same when it comes to map IRQs to set of CPUS. We need
> > > > > > to associate a resource (IRQ or address) to a set of cpus (or more in
> > > > > > general, masters).
> > > > > 
> > > > > Hrm.  See, I think I may be misunderstanding the problem again,
> > > > > because with this description I can see no problem.  It's already up
> > > > > to the device binding to describe the purpose of each entry in the reg
> > > > > property.  So what's the problem with it just being part of the
> > > > > binding to say which reg entry is associated with which master port?
> > > > 
> > > > That's exactly what we are trying to do. But to associate the reg
> > > > property to a master port we need a phandle, how can we pull that off
> > > > otherwise ?
> > > 
> > > Ah, right.  I was meaning that the binding specifies the abstract port
> > > number that each reg entry is associated with.  I'm fine with adding a
> > > property to map the port numbers to master devices on the other end.
> > > I think treating it in two steps like that is better than thinking of
> > > the master phandles being  directly associated with reg entries,
> > > because it also handles cases like having a bank of common/global
> > > registers not associated with any port/master, or cases where the
> > > ports aren't all identical and some need more resources than others.
> > 
> > That's fine by me. Can you provide me with an example of how the bindings
> > should tie a specific reg property to an abstract port number ? reg properties
> > ordering (ie index) ? Or port number encoded in the reg property itself ?
> 
> I think I agree with David (assuming I understand his argument
> correctly). Rather than putting the information into the single node, it
> is more consistent in DT terms to put that information into each of the
> 'users' of the device; so for each master, put in a property that
> associates it with a specific port. Something like this (completely off
> the top of my head; not at all fully thought out):
> 
> cpu at 0 {
> 	cci-port = <&cci1 0>;
> }
> 
> cpu at 1 {
> 	cci-port = <&cci1 1>;
> }
> 
> cpu at 2 {
> 	cci-port = <&cci1 2>;
> }
> 
> gpu at c8010000 {
> 	cci-port = <&cci1 3>;
> }

Nitpicking: the node above is not really a "master", it is a device tree
node representing GPU resources in the CPU address space since, if I got that
right, the device tree provides the address space view from a CPU perspective.

I am not saying this is right or wrong, just trying to make up my mind
myself to understand if this can cause any issues in the future.

> cci1: cci at c8000000 {
> 	reg = < ... >;
> }
> 
> An arguement against that would be that it is difficult to determine
> what all the bus-mastering devices are from looking at the cci node, but
> in real terms you would have a lookup table in the operating system
> /anyway/ for matching devices to ports and I would expect that as each
> device is discovered from the DT data the OS would populate it's lookup
> table.

Ok, I am not a big fan of this approach since it does not scale IMHO. I would
avoid adding a property to a cpu node every given time we need a device
requiring its resources to be linked to a (possibly set of) cpu(s).

If we add the property(ies) in the cci node, at least the changes become
self contained and the mechanism through which we link a resource to a
"master" becomes standard (phandle). All bits of info become available
through the cci node, but again, it is a choice to be made, I do not
know what's the best approach.

> It may be that Dave Martin's suggestion of using basically a dma-ranges
> type property with a phandle would be the correct interface. I've not
> thought enough about it yet to really get my head around it.

What I really really want to avoid is implementing something for CCI and then
realize that it does not work for other components with similar requirements.
With Mark and Dave Martin we tried to come up with a solution that can be
generalized, that was the main goal.

I will review again Dave Martin's suggestion and take into account your
suggestion (which is what David Gibson would like us to do if I got it
right, you formalized his proposal).

Thx,
Lorenzo