<br>
<br><tt><font size=2>Arnd Bergmann <arnd.bergmann@de.ibm.com> wrote
on 10/10/2006 08:31:11 AM:<br>
<br>
> On Monday 09 October 2006 22:44, Luke Browning wrote:<br>
> > How can we determine the number of installed spus to limit the
size of the <br>
> > gang? Should we just add a new check to spu_create(thread)
that fails <br>
> > when the number of spes threads in the gang is greater than the
number of <br>
> > installed spus.<br>
> <br>
> (cc: ing the mailing list, maybe someone has better ideas)<br>
> <br>
> I guess we could fail both the creating of threads in a gang<br>
> and the creation of nonschedulable contexts if there is not<br>
> enough space.<br>
> <br>
> That means we have to add housekeeping to the kernel for the<br>
> number of non-pinned SPUs that is available, as well as recording<br>
> the number of pinned SPUs in order to determine the maximum size<br>
> of a gang.<br>
</font></tt>
<br><tt><font size=2>Not sure what you mean by pinned-SPU? Is your response
specific to </font></tt>
<br><tt><font size=2>libspe 2.0 or 1.2?</font></tt>
<br>
<br><tt><font size=2>> <br>
> The simpler alternative would be that we simply allow the users to<br>
> specify impossible situations, e.g. pinning more than the physically<br>
> available SPUs or creating a gang that is larger than that number.<br>
> <br>
> As a result, the spu_run() call could simply block interruptibly,<br>
> until either the user kills the task or the condition becomes possible<br>
> when something else exits.</font></tt>
<br>
<br><tt><font size=2>This adds a lot of complexity to affinity scheduling
because we assume</font></tt>
<br><tt><font size=2>one spe thread per physical spu. We don't want
two spe threads to be</font></tt>
<br><tt><font size=2>queued to the same spu. </font></tt>
<br><tt><font size=2><br>
> <br>
> I'd prefer the simpler solution, but it's a tradeoff, because users<br>
> don't notice early. We should definitely not allow creating more<br>
> threads in a gang than what is physically possible, but that only<br>
> depends on the number of SPE nodes in the device tree and is easy<br>
> to determine.<br>
</font></tt>
<br><tt><font size=2>agreed. Are you recommending modifying spe_getgroup_max()
<spelling> </font></tt>
<br><tt><font size=2>in the library to return the number of 'available'
spus instead of </font></tt>
<br><tt><font size=2>the current hardcoded value (16). Note available does
not include </font></tt>
<br><tt><font size=2>security isolated spus.</font></tt>
<br>
<br><tt><font size=2>> <br>
> On a hypervisor system that does not have a fixed number of SPUs<br>
> assigned to a partition, we may need to have some way to communicate<br>
> the maximum number of SPUs to Linux through some hcall or device tree<br>
> entry.<br>
> <br>
</font></tt>
<br><tt><font size=2>Are you thinking of micro-partitioning of spu processors
here? Is </font></tt>
<br><tt><font size=2>anybody actually discussing that right now? I
hope not as we don't </font></tt>
<br><tt><font size=2>have a clear understanding of scheduling concurrency
dynamics yet in</font></tt>
<br><tt><font size=2>the new scheduler, ie. page fault resolution, locking,
... </font></tt>
<br>
<br><tt><font size=2>Luke</font></tt>