[SLOF] [PATCH slof] pci: Put non-prefetchable 64bit BARs into 32bit MMIO window

[Laurent Vivier]

On 25/04/2017 15:44, Alexey Kardashevskiy wrote:
> On 25/04/17 21:00, Laurent Vivier wrote:
>> On 25/04/2017 12:36, Laurent Vivier wrote:
>>> On 24/04/2017 05:01, Alexey Kardashevskiy wrote:
>>>> At the moment 64bit non-prefetchable BARs of devices behind PCI p2p
>>>> bridge go to a 64bit prefetchable windows which is not correct and
>>>> causes linux guests to fail to ioremap() such resources.
>>>>
>>>> This moves 64bit non-prefetchable BARs 32bit non-prefetchable window.
>>>>
>>>> Note that this does not make distinction between P2P and PHB so
>>>> from now on XHCI BARs will be allocated from 32bit MMIO space.
>>>> However since most 64bit-MMIO-capable devices have prefetchable BARs,
>>>> and XHCI BAR is just 4K (so it is unlikely to cause any space problems),
>>>> this should not affect usual behavior.
>>>>
>>>> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
>>>> ---
>>>>
>>>> This fixes QEMU's XHCI when it is put on a P2P PCI bridge.
>>>>
>>>> There is a little naming confusion as it may look like the patch
>>>> is changing assignment for all 64bit BAR but it does not as:
>>>> - "mmio" is used for non-prefetchable memory,
>>>> - "mem" is used for prefetchable memory.
>>>> ---
>>>>  slof/fs/pci-properties.fs | 6 +-----
>>>>  1 file changed, 1 insertion(+), 5 deletions(-)
>>>>
>>>> diff --git a/slof/fs/pci-properties.fs b/slof/fs/pci-properties.fs
>>>> index e6fd843..8594e5d 100644
>>>> --- a/slof/fs/pci-properties.fs
>>>> +++ b/slof/fs/pci-properties.fs
>>>> @@ -166,11 +166,7 @@
>>>>  \ Setup a non-prefetchable 64bit BAR and return its size
>>>>  : assign-mmio64-bar ( bar-addr -- 8 )
>>>>          dup pci-bar-size-mem64          \ fetch size
>>>> -        pci-next-mem64 @ 0 = IF          \ Check if we have 64-bit memory range
>>>> -	    pci-next-mmio
>>>> -	ELSE
>>>> -	    pci-next-mem64              \ for board-qemu we will use same range
>>>> -	THEN
>>>> +        pci-next-mmio
>>>>          assign-bar-value64              \ and set it all
>>>>  ;
>>>>  
>>>>
>>> Tested-by: Laurent Vivier <lvivier at redhat.com>
>>
>> This fixes the problem when the BAR is mem, not when it is io.
>>
>> I have the same problem as with XHCI as with virtio-blk devices behind a
>> pci bridge, and the error is:
>>
>> ...
>> PCI: Cannot allocate resource region 0 of device 0000:01:01.0, will remap
>> ...
>> virtio-pci 0000:01:01.0: can't enable device: BAR 0 [io  size 0x0040]
>> not assigned
>> ...
>>
>> And this patch doesn't fix it.
> 
> 
> Correct, it did not try to fix it. But afaict virtio uses IO in legacy
> drivers and also implements mmio BARs which are actually used in guest
> kernels we care about. Or there is a problem and something does not work?

You're right, adding "disable-legacy=on" to the virtio-blk-pci device
allows to use the disk and remove the errors from the guest kernel logs.

What I don't understand is why "disable-legacy=on" fixes the problem
even without your patch for SLOF...

Thanks,
Laurent