[Cbe-oss-dev] [PATCH 1/5] MARS: yield in workload queue wait busy loop

[Kazunori Asayama]

Arnd Bergmann wrote:
>> I worry that the probability of the SPE's stall may significantly 
>> increases because of use of mailboxes. We'll need to look into various 
>> possible scenarios and to evaluate performance impact if we try to use 
>> the mailboxes for this purpose. And we also need to consider about 
>> MARS's task switches since the MARS doesn't take care of MFC's status 
>> when switching tasks (yes, MARS's task switching is 'light weight'!); 
>> that is, if a task A writes anything to the SPE's mailbox then MARS 
>> switches the task A to another task B, the content of the mailbox still 
>> exists until some PPE thread reads it. That means one task can cause 
>> another task's stall even if they logically have no dependency.
> 
> I guess this basically means that the mailbox becomes useless for the
> MARS tasks, right? If so, it would be good to use it for synchronization

Yes, right. At least, some kind of trick, e.g. mailbox dispatcher or 
something, will be needed so that applications use the mailboxes with 
MARS (otherwise, the applications can't know which MARS task has sent 
the data via the mailboxes). Anyway, MARS has "data share queue" object 
to pass applications' data between PPE and SPEs so it will be acceptable 
to reserve the mailboxes for MARS's internal use.

> of the MARS kernel itself.
> Mailbox stalls should of course be avoided, but I suppose there is a way
> to combine synchronization through atomic memory updates with mailbox
> accesses to achieve this.

One possible solution is a combination of mailbox and futex as 
following. It is just an idea at this point.

   - Use one futex per synchronization object. Maybe the futex can be
     the first 32bit of the sync object and it also can be used for
     atomic operations for SPE-to-SPE and PPE-to-SPE communication.
   - A PPE thread waits for synchronization object by using the futex.
   - An SPE notifies the PPE a change of the synchronization object's
     status via its mailboxes. The EA of the futex is passed as the
     notification.
   - Use one PPE thread per SPE context to process the notifications.
     This thread has a loop which receives notifications (== futex's EA)
     via the mailboxes then wakes waiting threads up by using the futex.

-- 
(ASAYAMA Kazunori
   (asayama at sm.sony.co.jp))
t