[Cbe-oss-dev] Cell internal registers access

[Arnd Bergmann]

On Friday 05 May 2006 07:46, Benjamin Herrenschmidt wrote:
> While playing around with the RAS stuff and other bits, I've come
> accross an annoying thing. We have those gazillion of MMIO registers in
> the BE. They are grouped more/less into logical blocks, but there are
> some cross-overs (for example, the IOC_FIR is in the IIC block...). We
> have a bunch of files/drivers (pervasive, interrupts,...) who
> re-implement their own way of getting a list of BE's and mapping per-BE
> instance of these, doing it in different ways. We have various
> definitions of these, essentially as C structures, either in local .c
> files or .h files. It's a bit messy.

agreed.

> I'm tempted to create a common "module" (be_regs.c/h) to centralise that
> a bit more. I talked to Paulus a bit and he seems to agree with the
> approach. That is, put all the register block definitions and
> appropriate constants in the .h file, along with an enum of the various
> blocks. Then have the .c file, at boot, implement once and only once the
> algorithm of going through all BE's and mapping in all these things, and
> provide accessors for handing over a given mapped register block.
> 
> As the Cell platform maintainer, do you agree with this approach ? If
> yes, then I'll send a patch for doing it and adapting the existing
> modules in the upcoming couple of days that I'd like to have in 2.6.18
> and the RAS stuff will depend on it. (Typically, the RAS stuff need to
> peek at registers in a variety of different blocks, so the current
> approach of having the various register blocks defined and mapped
> locally doesn't work very well).

I'd really prefer having separate drivers for each subsystem of
the CPU that are able to handle all actions that are related to
their specific area. Now, for actually getting at the register areas,
there should probably be a common infrastructure, e.g.

void __iomem * get_cpu_io_area(int cpu, const char *name, unsigned long *size);

that can be called for pervasive, iic, ioc, ...

If we get the firmware to support a more generic system using one
SOC bus per physical CPU and device node under that for each area,
this method can also be made generic enough to support simple
on-chip mmio areas in general, not just limited to cell.

Therefore, I'd like to keep knowledge of what areas exist out
of the common part. My (idealized) concept would be that
for something that needs to touch multiple areas, a high-level
driver (oprofile, cpufreq, idle, ...) would do function calls
into the low-level driver (pervasive, interrupt, iommu, ...).
It may be overdesign, but I want to at least try to keep
that abstraction to see if it works out or not.

I had hoped that we could avoid a data structure specific to
a physical Cell BE and do all that with the existing CPU numbers
and NUMA node identifiers, but it seems we might need yet another
dimension for that.

However, I don't think that concept should be Cell platform
specific, I'm rather sure we will need something similar
for other designs with multi-core or SMT system-on-chip
characteristics.

> Note: I'm thinking about leaving the SPUs out of that since they are
> fairly separate and have already their own .h and exported accessors.
> It's more for IIC, BIU, IOC, PMD, etc...

As Christian Krafft now noticed while playing with some of the
thermal registers (you two should talk about that), we do need to
keep the SPEs in the picture, although in a different way: At least
the thermal registers have per-spe data in on of the mmio spaces
outside of the SPE-local (priv1/2- and problem-state) mmio areas.

So what is needed is, given a 'struct spu *', a way to find the
respective mmio area for PMD, and the index on the spu in that
area.

	Arnd <><