[Cbe-oss-dev] IOC translation & firmware question

[Benjamin Herrenschmidt]

Hi Hartmut, Christian, Mike !

Is the firmware doing any setting of the IOC cache or IOC translation at
all ? Is it left into the default power up state ? That is disabled ?

I've fully implemented the IOMMU support using Jeremy's work and
adapting it to my new stuff, and it seems to work fine. However, I'd
like to measure the performance impact of the IOMMU, so I have
implemented support for IOMMU off, in which case, I leave the IOC in the
state left by the firmware and pass direct memory pages addresses (with
just the top bit set for spider) to devices.

It seems to be working. The numbers I'm getting with a home brewed
network testing program (purely raw TCP read/write bandwidth) with the
current 2.6.19-rc4-git-of-the-day spidernet are:

 With IOMMU enabled on the sender : 54.5MB/sec
 With IOMMU off on the sender     : 58.0MB/sec

(in both tests, the receiver is running some standard SDK kernel, in
this case 2.6.16-be0623SDK.0.20060612)

Based on those numbers, I've decided to do like PowerMac, and disable
the IOMMU by default when there is less than 2Gb of RAM in the machine.

Note that for this to work properly, I'm assuming that when translation
is disabled, the BE is reacting as per defined in the CBEA architecture,
that is with M=1 and SO=3 (memory coherency required and full storage
ordering). There might be an issue if the IOIF "C" bit is not set to 1,
thus I expect the firmware to set it.

Is that a problem ? Should I force it to 1 myself ? Or is the firmware
setting it ? I haven't managed to find out where that bit is, in which
register, it's only mentioned in the CBEA documentation as far as I
know....

Cheers,
Ben.