[SLOF] [PATCH slof] pci: Put non-prefetchable 64bit BARs into 32bit MMIO window

Thomas Huth thuth at redhat.com
Thu Apr 27 17:50:36 AEST 2017


On 27.04.2017 09:10, Thomas Huth wrote:
> On 27.04.2017 08:54, Alexey Kardashevskiy wrote:
>> On Wed, 26 Apr 2017 15:23:40 +0200
>> Thomas Huth <thuth at redhat.com> wrote:
>>
>>> On 24.04.2017 05:01, Alexey Kardashevskiy wrote:
>>>> At the moment 64bit non-prefetchable BARs of devices behind PCI p2p
>>>> bridge go to a 64bit prefetchable windows which is not correct and
>>>> causes linux guests to fail to ioremap() such resources.
>>>>
>>>> This moves 64bit non-prefetchable BARs 32bit non-prefetchable
>>>> window.
>>>>
>>>> Note that this does not make distinction between P2P and PHB so
>>>> from now on XHCI BARs will be allocated from 32bit MMIO space.
>>>> However since most 64bit-MMIO-capable devices have prefetchable
>>>> BARs, and XHCI BAR is just 4K (so it is unlikely to cause any space
>>>> problems), this should not affect usual behavior.
>>>>
>>>> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
>>>> ---
>>>>
>>>> This fixes QEMU's XHCI when it is put on a P2P PCI bridge.
>>>>
>>>> There is a little naming confusion as it may look like the patch
>>>> is changing assignment for all 64bit BAR but it does not as:
>>>> - "mmio" is used for non-prefetchable memory,
>>>> - "mem" is used for prefetchable memory.
>>>> ---
>>>>  slof/fs/pci-properties.fs | 6 +-----
>>>>  1 file changed, 1 insertion(+), 5 deletions(-)
>>>>
>>>> diff --git a/slof/fs/pci-properties.fs b/slof/fs/pci-properties.fs
>>>> index e6fd843..8594e5d 100644
>>>> --- a/slof/fs/pci-properties.fs
>>>> +++ b/slof/fs/pci-properties.fs
>>>> @@ -166,11 +166,7 @@
>>>>  \ Setup a non-prefetchable 64bit BAR and return its size
>>>>  : assign-mmio64-bar ( bar-addr -- 8 )
>>>>          dup pci-bar-size-mem64          \ fetch size
>>>> -        pci-next-mem64 @ 0 = IF          \ Check if we have 64-bit
>>>> memory range
>>>> -	    pci-next-mmio
>>>> -	ELSE
>>>> -	    pci-next-mem64              \ for board-qemu we will
>>>> use same range
>>>> -	THEN
>>>> +        pci-next-mmio
>>>>          assign-bar-value64              \ and set it all
>>>>  ;  
>>>
>>> I'm sorry, but this is now causing trouble with the USB keyboard in
>>> *SLOF* instead. If I start QEMU now like this:
>>>
>>> qemu-system-ppc64 -nodefaults -serial mon:stdio \
>>>   -device pci-bridge,bus=pci.0,id=bridge1,chassis_nr=1 \
>>>   -device nec-usb-xhci,id=xhci1,bus=bridge1,addr=0x3 -vga std \
>>>   -device usb-kbd,bus=xhci1.0 -bios boot_rom.bin
>>>
>>> then SLOF complains: "usb-xhci: failed to initialize XHCI controller"
>>> and the keyboard is not working at the SLOF prompt anymore.
>>>
>>> Any idea why this is happening now?
>>
>>
>> XHCI's usb_hcd_dev::base == 0xc0000000 which is a bus address while it
>> is supposed to be a mapped address, i.e. 800000020000000.
>>
>> It used to work before the patch as pci address is the same as mapped
>> address - 210000000000 - and I am not sure if this is not an accident,
>> could be QEMU's PCI MMIO windows/spacing rework David did some time ago.
>>
>> I am trying to figure out now where usb_hcd_dev::base is set in SLOF,
>> it is tricky :-/
> 
> I think the value comes from usb-setup-hcidev in pci-class_0c.fs ... so
> it's likely that translate-my-address is failing now.
> 
> ... but thinking about this issue again, I now wonder whether your
> original patch was really the right solution here, since QEMU only
> provides three memory regions to the guest:
> 1) I/O
> 2) 32-bit memory
> 3) 64-bit memory
> There is no distinction between prefetchable and non-prefetchable here.
> So I think the original problem might have been caused by something else
> instead... (not sure about that yet, though)

Yup, looks like the real culprit is the "ranges" property of the PCI
bridges. This can not deal with 64-bit memory regions yet. I think we
should revert the patch to assign-mmio64-bar and fix
pci-bridge-range-props and friends instead...

 Thomas



More information about the SLOF mailing list