[Cbe-oss-dev] [RFC] [PATCH 0:8] SPU Gang Scheduling

[Arnd Bergmann]

On Tuesday 04 March 2008, Luke Browning wrote:
> Here's something to consider as a counter example.  If a higher priority
> gang is started, it must preempt the lower priority gang.  The preempted
> gang might have some contexts in user mode and some in kernel mode (ie.
> the spus are acutally running).  In this case, the preempted gang has to
> be put on the runqueue, so that the contexts that were stopped and
> unloaded may be loaded and started again automatically.  Otherwise, the
> application would hang.  

It would not hang indefinitely, but only until all the contexts have
resolved their callbacks and page faults and want to run again.
One case where it would indeed hang is when one thread makes a callback
to the PPU that ends up waiting for another thread in the same gang
to perform a specific action on the SPU before returning to spu_run.
With your current code, that is allowed, while with the model I'm
suggesting it would be a bug in the application.

Do you see that as a significant problem? With gang scheduling, there
are already enough ways for an application to deadlock itself, so
I don't have a problem with another one, as long as it's reasonable
and well-documented.

> > The reason for that  is that running the gang when  only one of the
> > threads in it would be somewhat unfair to  the other sleeping gangs
> > in the system that have all contexts in a thread trying  to run SPU
> > code.
> 
> You could add heuristics to handle unfairness by processing N elements
> on the runqueue instead of just getting the next one.  Maybe look at
> three or four gangs at the same priority and chose the one with the
> highest percentage of contexts in the spu_run.

I'd like to avoid heuristics like this but rather half the behavior of
the kernel predictable and obvious.

One potential option I can see is to actually load a half-idle gang if 
there is nothing else that is trying to run, i.e. when all gangs in the
system are half-idle, but I would like to avoid that complexity.

	Arnd <><