[Cbe-oss-dev] spu question

[Arnd Bergmann]

On Tuesday 10 October 2006 15:09, Luke Browning wrote:
> 
> > I guess we could fail both the creating of threads in a gang
> > and the creation of nonschedulable contexts if there is not
> > enough space.
> > 
> > That means we have to add housekeeping to the kernel for the
> > number of non-pinned SPUs that is available, as well as recording
> > the number of pinned SPUs in order to determine the maximum size
> > of a gang.
> 
> Not sure what you mean by pinned-SPU? Is your response specific to 
> libspe 2.0 or 1.2?

The spufs-create-isolated-2.diff patch adds the concept of
nonscheduleable SPUs as a prerequisite for isolated SPUs.

This is not specific to a libspe version, but is a new feature
of spufs.

> > The simpler alternative would be that we simply allow the users to
> > specify impossible situations, e.g. pinning more than the physically
> > available SPUs or creating a gang that is larger than that number.
> > 
> > As a result, the spu_run() call could simply block interruptibly,
> > until either the user kills the task or the condition becomes possible
> > when something else exits.
> 
> This adds a lot of complexity to affinity scheduling because we assume
> one spe thread per physical spu.  We don't want two spe threads to be
> queued to the same spu. 

I don't understand. Where do you want to queue an spu_context if all
spus arlready have a context loaded, coming from another gang?

> > I'd prefer the simpler solution, but it's a tradeoff, because users
> > don't notice early. We should definitely not allow creating more
> > threads in a gang than what is physically possible, but that only
> > depends on the number of SPE nodes in the device tree and is easy
> > to determine.
> 
> agreed. Are you recommending modifying spe_getgroup_max() <spelling> 
> in the library to return the number of 'available' spus instead of 
> the current hardcoded value (16).

I think we finally did that in a recent libspe change. At least, it
has been planned to be done for a long time. Since you bring it up
again, I noticed that we may have a problem 

> Note available does not include security isolated spus.

My point was exactly the opposite, I just wouldn't worry about these,
if you create a gang with more threads than currently available, but
less than physically there (e.g. one is in isolation mode), I'd just
wait for the isolated one to exit.

> > On a hypervisor system that does not have a fixed number of SPUs
> > assigned to a partition, we may need to have some way to communicate
> > the maximum number of SPUs to Linux through some hcall or device tree
> > entry.
> > 
> 
> Are you thinking of micro-partitioning of spu processors here? Is 
> anybody actually discussing that right now?  I hope not as we don't 
> have a clear understanding of scheduling concurrency dynamics yet in
> the new scheduler, ie. page fault resolution, locking, ...

I'm thinking of the currently implemented hypervisors that let different
operating systems allocate SPUs dynamically.

	Arnd <><