[Cbe-oss-dev] spu question

[Luke Browning]

Arnd Bergmann <arnd.bergmann at de.ibm.com> wrote on 10/10/2006 08:31:11 AM:

> On Monday 09 October 2006 22:44, Luke Browning wrote:
> > How can we determine the number of installed spus to limit the size of 
the 
> > gang?   Should we just add a new check to spu_create(thread) that 
fails 
> > when the number of spes threads in the gang is greater than the number 
of 
> > installed spus.
> 
> (cc: ing the mailing list, maybe someone has better ideas)
> 
> I guess we could fail both the creating of threads in a gang
> and the creation of nonschedulable contexts if there is not
> enough space.
> 
> That means we have to add housekeeping to the kernel for the
> number of non-pinned SPUs that is available, as well as recording
> the number of pinned SPUs in order to determine the maximum size
> of a gang.

Not sure what you mean by pinned-SPU? Is your response specific to 
libspe 2.0 or 1.2?

> 
> The simpler alternative would be that we simply allow the users to
> specify impossible situations, e.g. pinning more than the physically
> available SPUs or creating a gang that is larger than that number.
> 
> As a result, the spu_run() call could simply block interruptibly,
> until either the user kills the task or the condition becomes possible
> when something else exits.

This adds a lot of complexity to affinity scheduling because we assume
one spe thread per physical spu.  We don't want two spe threads to be
queued to the same spu. 

> 
> I'd prefer the simpler solution, but it's a tradeoff, because users
> don't notice early. We should definitely not allow creating more
> threads in a gang than what is physically possible, but that only
> depends on the number of SPE nodes in the device tree and is easy
> to determine.

agreed. Are you recommending modifying spe_getgroup_max() <spelling> 
in the library to return the number of 'available' spus instead of 
the current hardcoded value (16). Note available does not include 
security isolated spus.

> 
> On a hypervisor system that does not have a fixed number of SPUs
> assigned to a partition, we may need to have some way to communicate
> the maximum number of SPUs to Linux through some hcall or device tree
> entry.
> 

Are you thinking of micro-partitioning of spu processors here? Is 
anybody actually discussing that right now?  I hope not as we don't 
have a clear understanding of scheduling concurrency dynamics yet in
the new scheduler, ie. page fault resolution, locking, ... 

Luke
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/cbe-oss-dev/attachments/20061010/2fd66c3f/attachment.htm>