[SLOF] [PATCH slof] pci: Put non-prefetchable 64bit BARs into 32bit MMIO window

[Alexey Kardashevskiy]

On 26/04/17 13:28, Alexey Kardashevskiy wrote:
> On 25/04/17 01:29, Thomas Huth wrote:
>> On 24.04.2017 05:01, Alexey Kardashevskiy wrote:
>>> At the moment 64bit non-prefetchable BARs of devices behind PCI p2p
>>> bridge go to a 64bit prefetchable windows which is not correct and
>>> causes linux guests to fail to ioremap() such resources.
>>>
>>> This moves 64bit non-prefetchable BARs 32bit non-prefetchable window.
>>>
>>> Note that this does not make distinction between P2P and PHB so
>>> from now on XHCI BARs will be allocated from 32bit MMIO space.
>>
>> I wonder why this was working when the XHCI controller was connected
>> directly to the PHB? Do the memory regions behave differently here?
> 
> 
> PCI bridges have IO, 32bit prefetchable, 32bit non-prefetchable, 64bit
> prefetchable windows; they simply do not have space for storing 64bit
> non-prefetchable window address in the config space.
> 
> sPAPR PHB bridges have IO, 32bit non-prefetchable, 64bit non-prefetchable
> windows - b_p(1) is not set (why is that I am not so sure, it has been
> always like that, may have something to do with prefetching support on the
> hardware actual by the time).
> 
> Prefetchable resources can be allocated within non-prefetchable windows but
> not vice versa.

Ah, it is also worth mentioning that BAR allocation done by SLOF and by
skiboot is different - on the bare metal, XHCI with 64bit nonprefetchable
BARs is put into 32bit MMIO window already.


> 
> 
> 
>>
>>> However since most 64bit-MMIO-capable devices have prefetchable BARs,
>>> and XHCI BAR is just 4K (so it is unlikely to cause any space problems),
>>> this should not affect usual behavior.
>>>
>>> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
>>> ---
>>>
>>> This fixes QEMU's XHCI when it is put on a P2P PCI bridge.
>>>
>>> There is a little naming confusion as it may look like the patch
>>> is changing assignment for all 64bit BAR but it does not as:
>>> - "mmio" is used for non-prefetchable memory,
>>> - "mem" is used for prefetchable memory.
>>> ---
>>>  slof/fs/pci-properties.fs | 6 +-----
>>>  1 file changed, 1 insertion(+), 5 deletions(-)
>>>
>>> diff --git a/slof/fs/pci-properties.fs b/slof/fs/pci-properties.fs
>>> index e6fd843..8594e5d 100644
>>> --- a/slof/fs/pci-properties.fs
>>> +++ b/slof/fs/pci-properties.fs
>>> @@ -166,11 +166,7 @@
>>>  \ Setup a non-prefetchable 64bit BAR and return its size
>>>  : assign-mmio64-bar ( bar-addr -- 8 )
>>>          dup pci-bar-size-mem64          \ fetch size
>>> -        pci-next-mem64 @ 0 = IF          \ Check if we have 64-bit memory range
>>> -	    pci-next-mmio
>>> -	ELSE
>>> -	    pci-next-mem64              \ for board-qemu we will use same range
>>> -	THEN
>>> +        pci-next-mmio
>>>          assign-bar-value64              \ and set it all
>>>  ;
>>
>> I think this patch is right. Using prefetchable memory here was
>> certainly a bad idea, and if I get the PCI-to-PCI bridge spec right,
>> there is no such thing like a dedicated 64-bit non-prefetchable memory
>> range, so using the non-prefetchable 32-bit memory range is the only
>> option here.
>>
>> Reviewed-by: Thomas Huth <thuth at redhat.com>
> 
> Thanks! I'll push it now.
> 
> 


-- 
Alexey