[Skiboot] [PATCH 01/11] xive/p9: Introduce XIVE_INT_ORDER

[Gustavo Romero]

On 6/8/20 3:25 AM, Cédric Le Goater wrote:
> On 6/8/20 5:29 AM, Gustavo Romero wrote:
>> Hi Cédric,
>>
>> On 6/4/20 10:21 AM, Cédric Le Goater wrote:
>>> The size of the interrupt number space is constrained by the block and
>>> index fields of the trigger data exchanged between source units and
>>> the XIVE IC. These are respectively 4 and 28 bits, which gives us a 32
>>> bits interrupt number space. But the XICS emulation requires 8 bits to
>>> encode the CPPR value. The system interrupt number space is therefore
>>> constrained to 24 bits and on a chip, to 20 bits because the XIVE
>>> driver configures the HW to use one block per chip.
>>>
>>> XIVE_INT_ORDER defines the size of the interrupt number space : 1M per
>>> chip.
>>>
>>> To control these interrupts, the driver defines in the VC BAR of the
>>> controller a range of 384G of ESB pages giving access to 3M interrupts.
>>> The VSD for the memory table is smaller than the index and accesses to
>>> some ESB pages are not backed by a memory table structure. If such an
>>> access occurred, it would result in a FIR.
>>>
>>> It never happened but this is something to fix with a finer configuration
>>> of the VC BAR.
>>>
>>> Signed-off-by: Cédric Le Goater <clg at kaod.org>
>>> ---
>>>    hw/xive.c | 32 ++++++++++++++++++--------------
>>>    1 file changed, 18 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/hw/xive.c b/hw/xive.c
>>> index 9a36f1ab2111..b9cdc4b0e317 100644
>>> --- a/hw/xive.c
>>> +++ b/hw/xive.c
>>> @@ -146,18 +146,22 @@
>>>     * so we could potentially make the IVT size twice as big, but for now
>>>     * we will simply share it and ensure we don't hand out IPIs that
>>>     * overlap the HW interrupts.
>>> + *
>>> + * TODO: adjust the VC BAR range for IPI ESBs on this value
>>>     */
>>> -#define MAX_INT_ENTRIES        (1 * 1024 * 1024)
>>> +
>>> +#define XIVE_INT_ORDER        20 /* 1M interrupts */
>>> +#define XIVE_INT_COUNT        (1ul << XIVE_INT_ORDER)
>>>      /*
>>>     * First interrupt number, also the first logical interrupt number
>>> - * allocated by Linux
>>> + * allocated by Linux (the first numbers are reserved for ISA)
>>>     */
>>>    #define XIVE_INT_FIRST        0x10
>>>      /* Corresponding direct table sizes */
>>> -#define SBE_SIZE    (MAX_INT_ENTRIES / 4)
>>> -#define IVT_SIZE    (MAX_INT_ENTRIES * 8)
>>> +#define SBE_SIZE            (XIVE_INT_COUNT / 4)
>>> +#define IVT_SIZE            (XIVE_INT_COUNT * 8)
>>
>> I thought VC BAR would configure ESB (aka SBE) plus END, not IVT (aka EAS
>> or EAST), since the other FSM (related to ESn and ESe) is associated with
> 
> what's an FSM ?

Pardon me, I should not have introduced yet another acronym in the xive
context (we have plenty of them already). I just meant the finite state machine
associated with END processing.


>> END and not with EAS, plus END size is 8 bytes and EAS size is 2 bytes.
> 
> The maximum number of interrupts XIVE_INT_COUNT directly controls the size
> of the PQ bits (SBE) table in RAM and the size of EAS (IVE on P9) table.

I see. And a note correcting myself above: I meant "words" instead of "bytes".


> The size of the END (EQD on P9) table is controlled differently.
> 
> The VC BAR configuration is done elsewhere. It exposes the ESB pages to
> control the PQ bits of the standard interrupts and the PQ bits in the
> ENDs.

Yes, that's the configuration I had in mind. Thanks for clarifying that
the IVT_SIZE defined above is not related to the END size set up in the
early VC BAR configuration step.

In (XIVE_INT_COUNT * 8) I think indeed it will help to make the amount
of 8 bytes more clear if it's determined from an sizeof() IVE/EAS struct,
which size is u64 bits, or 2 *words* as per the docs.

Thanks for the explanations.


Best regards,
Gustavo