[Cbe-oss-dev] [RFC] [PATCH 0:8] SPU Gang Scheduling

[Luke Browning]

On Tue, 2008-03-04 at 05:51 +0100, Arnd Bergmann wrote:
> On Tuesday 04 March 2008, Luke Browning wrote:
> > Finally, this means that a gang is eligible to be run as long as 
> > one context in the gang is runnable.  Major page faulting is the other 
> > event that may cause a gang to be preempted.  It is implemented via a
> > nfaulting count and a call to yield.  In this case, the gang needs to
> > be put on the runqueue as the context is in kernel mode.  It is sort of 
> > a step down scheduling technique to give something else a chance to run.
> 
> I think the last time we talked about this, the conclusion was that
> we should always do lazy loading as well as unloading of the entire
> gang, meaning that it only gets started as long as *all* threads in
> the gang actually try to run on the SPU, and then run until the end
> of their time slice or until the point  where all threads return to
> the PPU by means of stop-and-signal or page faults.

That would be more complicated, if I understand you correctly.  Start
requires all in spu_run, but once it got going the gang would be allowed
to run with some of the contexts outside of spu_run at least for a
while.  spu_run would have to run in two modes.

Here's something to consider as a counter example.  If a higher priority
gang is started, it must preempt the lower priority gang.  The preempted
gang might have some contexts in user mode and some in kernel mode (ie.
the spus are acutally running).  In this case, the preempted gang has to
be put on the runqueue, so that the contexts that were stopped and
unloaded may be loaded and started again automatically.  Otherwise, the
application would hang.  

> 
> The reason for that  is that running the gang when  only one of the
> threads in it would be somewhat unfair to  the other sleeping gangs
> in the system that have all contexts in a thread trying  to run SPU
> code.

You could add heuristics to handle unfairness by processing N elements
on the runqueue instead of just getting the next one.  Maybe look at
three or four gangs at the same priority and chose the one with the
highest percentage of contexts in the spu_run.

Luke