PCI hotplug + EEH issues

[Todd Inglett]

On Mon, 2002-09-09 at 08:01, Anton Blanchard wrote:
>
> Hi,
>
> Ive almost finished a PCI hotplug driver that works on SMP and LPAR.
> There is a bit more work to be done to support DLPAR (ie PCI-PCI bridges
> appearing and disappearing), but the basics seem to work OK.

Cool!  Do you have a patch?  I'd like to see this working :)

> One thing that caused problems is the faking up of the device tree node
> for function 0 of a PCI-PCI bridge when its not in our partition. I no
> longer do that in my local tree, Ive also fixed things to not require
> that fake device node.

I've hated this too.  There's got to be a better way to handle it, but
I'm guessing is that you moved the logic into the Linux pci read funcs?
I didn't want to special case that code, but I guess it doesn't need to
be fast either.

> Ive also been experimenting with reworking the EEH enable code. We
> should walk all devices and enable EEH before we do the bus walk,
> otherwise we touch the devices with EEH disabled.

I have a fix coded for bug 1197.  I'll attach the patch below.

> One question I had was why we are using device nodes everywhere (like
> pci config read/write)? With the recent changes in 2.5, it is harder
> to get back to device node and with a few small changes I no longer
> need to convert pci_dev to device node. (except for the few times at
> boot when we need to fake config reads).

The main reason was to prevent Linux from accessing devices which have
failed (i.e. BIST).  I also learned last week about the OF "status"
property that we are supposed to honor, and ironically I had just
deleted that bit of code 2 weeks before.  I have that coded and ready to
check in.  It's also just a few lines of code.

So if Linux wants to touch a device that is dead, we need to intercept
it and return bad status.  This is on the config reads, of course.
There are other ways to "flag" this but if we are going to hang arch
specific data in the pci_dev it may as well be the device node.  And if
we are going to need to support early PCI config access (i.e. BIST) we
may as well use the device node as the device "handle" for the config
reads.  So they naturally fell together.

What's changed in 2.5 that has made this harder?  I agree that Linux
fights with connecting arch data into the pci_dev when that arch data is
needed for the config reads.  A more natural approach would be to
abandon the pci driver bus walk and do our own by manufacturing the
pci_dev/bus tree from the device tree.  In fact, we could create the
entire pci_dev tree without doing a single config read!

I've been meaning to try coding this to see how it goes but haven't
found the time.  I'm also not sure what function we would lose nor am I
sure how much smaller the code will get.  We'd of course be dependent on
firmware getting it right, but so far I haven't found any counter
examples -- even with unsupported adapters.  Thoughts?

> Speaking of which I had a look and the current LPAR behaviour (function
> 0 not implemented) is in violation of the PCI spec.

True...but it would be hard to re-wire the systems in the field :(.  I
suppose RTAS could work around it like we do.

Personally, I'd like to rework a lot about the way we do eeh.  For
instance, I don't think we need to encode the mmio addresses since it
doesn't appear we will get a faster way to query eeh status, nor do we
get false positives anyway (at least from the little instrumenting code
I have in there I haven't seen any).  Since false positives never/rarely
happen we can code up handling of an eeh failure to be as slow as we
want  And the RTAS call will guarantee slowness anyway :).

-todd

-------------- next part --------------
A non-text attachment was scrubbed...
Name: eeh-init.patch
Type: text/x-diff
Size: 3536 bytes
Desc: not available
Url : http://ozlabs.org/pipermail/linuxppc64-dev/attachments/20020909/42fea718/attachment.patch