[Skiboot] [PATCH 2/2] phb4: set PBCQ Tunnel BAR for tunneled operations

[Philippe Bergheaud]

On 13/11/2017 12:19, Benjamin Herrenschmidt wrote:
> On Mon, 2017-11-13 at 11:45 +0100, Philippe Bergheaud wrote:
>> On 07/11/2017 21:18, Benjamin Herrenschmidt wrote:
>>> On Tue, 2017-11-07 at 14:19 +0100, Philippe Bergheaud wrote:
>>>> I understand that the Tunnel BAR value is chosen by the device logic,
>>>> within the regular BARs assigned to the device at power-on (an entire
>>>> BAR, or a subset). The Tunnel BAR value is fetched by the device driver,
>>>> that passes it to skiboot, to set the PBCQ Tunnel BAR Response register.
>>>>
>>>> I think that this patch is required. I cannot see how to reverse the
>>>> information flow.
>>> Why would it be chosen by the device ? I don't understand...
>>>
>> There can be more than one device connected to the PHB. But as there
>> is only one PBCQ Tunnel BAR Response register, only one device will
>> be able to use tunneled operations at a certain time.
> Why ? I don't understand. What is special about those tunneled
> operations that require that sort of mutual exclusion ?

The reason for the mutual exclusion is the unicity of the PBCQ Tunnel
BAR Response register, that indicates the address where responses are
expected by the device that has initiated the operation.

>>   I have chosen
>> a first-come first-serve policy, where a device will:
>>
>> 1. Acquire tunneled operations the by setting the register with
>>      its own specific BAR value, chosen within the range of its BARs,
>>      where tunneled operation responses will be expected.
> Ok I obsiously don't understand how tunneled operations work. Can you
> elaborate a bit ?

There are two kinds of tunneled operations: AS notify and atomics.
AS notify is not guaranted to succeed. The host must tell the device
whether the target thread has been woken up or not.

Not all atomic operations require a response. But Compare and Swap,
for example, does: the device must know the status of the comparison,
as swap is only performed in case of success.

>> 2. Release tunneled operations after use by resetting the PBCQ Tunnel
>>      BAR response register, authenticating itself by passing its device-
>>      specific BAR value again (as required by the API).
>>
>> The register value indicates which device should receive the tunneled
>> operation responses. The value of the register must be dynamically
>> modified by devices to point to their own MMIO range. It cannot be
>> statically set by skiboot.
> Oh so basically we expect a "response" to the device ? That smells like
> a disgusting HW hack to me ...
>

Yes, limiting tunneled operations to one device per PHB seems strange.
I have arbitrarily chosen the first-come first-serve policy to cope
with this limitation.

Philippe