[openib-general] Re: [PATCH 07/16] ehca: interrupt handling routines

[Shirley Ma]

openib-general-bounces at openib.org wrote on 05/09/2006 09:49:19 AM:

> Quoting r. Roland Dreier <rdreier at cisco.com>:
> > The trivial way to do it would be to use the same idea as the current
> > ehca driver: just create a thread for receive CQ events and a thread
> > for send CQ events, and defer CQ polling into those two threads.

I have done some patch like that on top of splitting CQ. The problem I 
found that hardware interrupt favors one CPU. Most of the time these two 
threads are running on the same cpu according to my debug output. You can 
easily find out by cat /proc/interrupts and /proc/irq/XXX/smp_affinity. 
ehca has distributed interrupts evenly on SMP, so it gets the benefits of 
two threads, and gains much better throughputs.

The interesting thing is the UP results are much better than SMP results 
with this approach on mthca.

> For RX, isn't this basically what NAPI is doing?
> Only NAPI seems better, avoiding interrupts completely and avoiding 
> latency hit
> by only getting triggered on high load ...
> 
> -- 
> MST

According to some results from different resouces, NAPI only gives 3%-10% 
performance improvement on single CQ.
I am trying a simple NAPI patch on splitting CQ now to see how much 
performance there.

Thanks
Shirley Ma
IBM Linux Technology Center
15300 SW Koll Parkway
Beaverton, OR 97006-6063
Phone(Fax): (503) 578-7638
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20060509/f67ee67c/attachment.htm>