default IRQ affinity change in v2.6.27 (breaking several SMP PPC based systems)

[Kumar Gala]

On Oct 24, 2008, at 11:09 AM, Chris Snook wrote:

> Kumar Gala wrote:
>> On Oct 24, 2008, at 10:17 AM, Chris Snook wrote:
>>> Kumar Gala wrote:
>>>> It appears the default IRQ affinity changes from being just cpu 0  
>>>> to all cpu's.  This breaks several PPC SMP systems in which only  
>>>> a single processor is allowed to be selected as the destination  
>>>> of the IRQ.
>>>> What is the right answer in fixing this?  Should we:
>>>>   cpumask_t irq_default_affinity = 1;
>>>> instead of
>>>>   cpumask_t irq_default_affinity = CPU_MASK_ALL?
>>>
>>> On those systems, perhaps, but not universally.  There's plenty of  
>>> hardware where the physical topology of the machine is abstracted  
>>> away from the OS, and you need to leave the mask wide open and let  
>>> the APIC figure out where to map the IRQs.  Ideally, we should  
>>> probably make this decision based on the APIC, but if there's no  
>>> PPC hardware that uses this technique, then it would suffice to  
>>> make this arch-specific.
>> What did those systems do before this patch?  Its one thing to  
>> expose a mask in the ability to change the default mask in /proc/ 
>> irq/default_smp_affinity.  Its another (and a regression in my  
>> opinion) to change the mask value itself.
>
> Before the patch they took an extremely long time to boot if they  
> had storage attached to each node of a multi-chassis system,  
> performed poorly unless special irqbalance hackery or manual  
> assignment was used, and imposed artificial restrictions on the  
> granularity of hardware partitioning to ensure that CPU 0 would  
> always be a CPU that could service all interrupts necessary to boot  
> the OS.
>
>> As for making it ARCH specific, that doesn't really help since not  
>> all PPC hw has the limitation I spoke of.  Not even all MPIC (in  
>> our cases) have the limitation.
>
> What did those systems do before this patch? :)
>
> Making it arch-specific is an extremely simple way to solve your  
> problem without making trouble for the people who wanted this patch  
> in the first place.  If PPC needs further refinement to handle  
> particular *PICs, you can implement that without touching any arch- 
> generic code.


So why not just have x86 startup code set irq_default_affinity =  
CPU_MASK_ALL than?

- k