[Cbe-oss-dev] Cell internal registers access

[Benjamin Herrenschmidt]

> Therefore, I'd like to keep knowledge of what areas exist out
> of the common part. My (idealized) concept would be that
> for something that needs to touch multiple areas, a high-level
> driver (oprofile, cpufreq, idle, ...) would do function calls
> into the low-level driver (pervasive, interrupt, iommu, ...).
> It may be overdesign, but I want to at least try to keep
> that abstraction to see if it works out or not.

So I would add a get_xxx_FIR() for every xxx sub_module ? Hrm .. the RAS
stuff is a bit annoying there. I don't know.. I don't beleive in too
much abstractions. I'm ok keeping the actual definitions of the register
spaces and associated constants in separate files, but I'm not sure we
should absolutely keep them private to some "driver" for that module.

If I follow your logic, for RAS, just in order to get to the FIR's (and
maybe a couple others), I would need to create a module for the MIC, a
module for the IOC, a module for the BIU, etc... 

I think the actual "driver" modules in the system slightly overlap
physical cell modules.
 
> I had hoped that we could avoid a data structure specific to
> a physical Cell BE and do all that with the existing CPU numbers
> and NUMA node identifiers, but it seems we might need yet another
> dimension for that.

I'm not sure what you mean in the above. There is a data structure for
the PMD (bad name btw :), one for hte IIC (missing the IOC_FIR btw),
there will be one for the MIC, etc... Then, my idea was to have a module
that hands out those to clients. This module is the one to know the
thread<->BE mapping etc... Clients call it with something like

struct cbe_xxx_regs __iomem *cbe_get_xxx_registers(int cpu);

That modules contains the thread<->BE mapping logic, the device-tree
parsing (thus if we ever clean that device-tree and remove those hackish
address properties, it's the only thing that will need to change) etc...

In fact, I'm tempted to also provide lower level BE<->thread mapping
routines in there for the few cases that might be interested.

> However, I don't think that concept should be Cell platform
> specific, I'm rather sure we will need something similar
> for other designs with multi-core or SMT system-on-chip
> characteristics.

The SMT <-> core mapping isn't quite well defined... Cell uses
ibm,interrupt-server#s but I'm not sure I completely like it at this
point... At least I'm not ready to recommend it to other embedded
vendors. Still need to think about it. In the meantime, this is not a
blocking issue as long as we don't have that logic copied in every
module, which is the case today.

> > Note: I'm thinking about leaving the SPUs out of that since they are
> > fairly separate and have already their own .h and exported accessors.
> > It's more for IIC, BIU, IOC, PMD, etc...
> 
> As Christian Krafft now noticed while playing with some of the
> thermal registers (you two should talk about that), we do need to
> keep the SPEs in the picture, although in a different way: At least
> the thermal registers have per-spe data in on of the mmio spaces
> outside of the SPE-local (priv1/2- and problem-state) mmio areas.

They do but that isn't a problem. I'm really only talking about
definition the register maps for the various modules and getting access
to them. Regarding SPE thermal management, we simply don't know yet what
we want to do. Wether the SPE layer will be the one to query the
temperatures or wether we'll have a temperature management module that
will call into the SPE layer. That is not a problem though. The SPE
termperatures are referenced by physical SPE number on a given BE. The
spu_base code already knows how to map between physical SPEs, BEs
etc...  

> So what is needed is, given a 'struct spu *', a way to find the
> respective mmio area for PMD, and the index on the spu in that
> area.

Well, if needed, we can have the "registers" module described above have
an API that takes a physical BE number (since that's what SPU layer
wants) and have a cbe_thread_to_be() call for callers who only have an
smp_processor_id().

Ben.