device not available because of BAR 0 collisions

[Steven A. Falco]

On 04/26/2011 07:39 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2011-04-26 at 09:38 -0400, Steven A. Falco wrote:
>> On 04/25/2011 08:01 PM, Benjamin Herrenschmidt wrote:
>>> On Mon, 2011-04-25 at 16:10 -0400, Steven A. Falco wrote:
>>>> I'm getting an error message when trying to talk to some custom
>>>> hardware:
>>>>
>>>> dx83xx 0001:43:00.0: device not available because of BAR 0
>>>> [0xa1000000-0xa1ffffff] collisions
>>>>
>>>> I see in setup-res.c that this message comes out when there is no
>>>> parent for
>>>> a device resource.
>>>
>>>  .../...
>>>
>>> It mostly happens in arch/powerpc/kernel/pci-common.c and the generic
>>> setup-res.c
>>>
>>> Try #define DEBUG at the top (before the #includes) of pci-common.c and
>>> pci_32.c (remove the exiting #undef in the last one) and send us the
>>> full dmesg log, along with the output of cat /proc/iomem
> 
> Have you set any specific flags ? IE. Modified the value of
> ppc_pci_flags from what the 4xx code sets originally ?

For fun, I just tried changing:

ppc_pci_set_flags(PPC_PCI_REASSIGN_ALL_RSRC);

to:

ppc_pci_set_flags(PPC_PCI_PROBE_ONLY);

I realize that is the exact opposite of what you were suggesting, but
please bear with me for a bit.

I also changed the PCIE 0 ranges from:

ranges = <0x02000000 0x00000000 0x80000000 0x90000000 0x00000000 0x10000000
          0x01000000 0x00000000 0x00000000 0xe8010000 0x00000000 0x00010000>;

ranges = <0x02000000 0x00000000 0x90000000 0x90000000 0x00000000 0x10000000
          0x01000000 0x00000000 0x00000000 0xe8010000 0x00000000 0x00010000>;

I changed the ranges not because I wanted a 1:1 map, but because 90000000 is
what U-Boot chooses when it scans PCIe 1.

At this point, everything is working.  Here is /proc/iomap:

90000000-9fffffff : /plb/pciex at 0c0000000
  90000000-94ffffff : PCI Bus 0001:41
    90000000-9001ffff : 0001:41:00.0
    90100000-94ffffff : PCI Bus 0001:42
      90100000-92ffffff : PCI Bus 0001:43
        91000000-91ffffff : 0001:43:00.0  //<--- was missing before
        92000000-92ffffff : 0001:43:00.0  //<--- was missing before
      93000000-94ffffff : PCI Bus 0001:44
        93000000-93ffffff : 0001:44:00.0  //<--- was missing before
        94000000-94ffffff : 0001:44:00.0  //<--- was missing before
e0000000-e7ffffff : /plb/pciex at 0a0000000
  e0000000-e7ffffff : PCI Bus 0000:01
    e0000000-e00fffff : 0000:01:00.0
    e0100000-e01fffff : 0000:01:00.0
    e4000000-e7ffffff : 0000:01:00.0
ef600200-ef600207 : serial
ef600300-ef600307 : serial
ef600600-ef600606 : spi_ppc4xx_of
ef6c0000-ef6cffff : dwc_otg.0
  ef6c0000-ef6cffff : dwc_otg
fc000000-ffffffff : fc000000.nor_flash

Now I see the bars for the ASICs (flagged above).  I could stop here,
and declare success, but I don't really like this solution, because it
requires me to be sure the dts has the same bus addresses that U-Boot
will choose.  Seems risky.

Tentative conclusion:  Either I still have something set wrong in my dts
or there is a bug in the Linux PCI bus mapping code.

	Steve

> 
> It does look to me like some of your device BARs have been setup already
> by the firmware in a way that conflict with the way you configure your
> ranges, and the kernel doesn't appear to detect nor try to remap that
> which would happen if you have the "probe only" flag set.
> 
> IE. On your c0000000 bus, you have memory at 90000000 CPU space mapped
> to 80000000 PCI space. However, when probing, the kernel finds:
> 
> pci 0001:41:00.0: reg 10 32bit mmio: [0x90000000-0x9001ffff]
> 
> IE. A BAR was already set with a value of 90000000 PCI-side which is out
> of the bounds you have for your bus.
> 
> Maybe you really want to configure that second bus to have CPU 90000000
> mapped to 90000000 PCI-side ? (IE. a 1:1 mapping). That would be
> something to fix in your "ranges" property.
> 
> Cheers,
> Ben.


-- 
A: Because it makes the logic of the discussion difficult to follow.
Q: Why shouldn't I top post?
A: No.
Q: Should I top post?