[PATCH 0/4][RFC] lro: Generic Large Receive Offload for TCP traffic

[Andrew Gallatin]

Jan-Bernd Themann wrote:
> Hi,
> 
> this patch set contains the latest generic LRO code, a Kconfig / Makefile
> and an eHEA patch demonstrating how the "aggregate SKB" interface has to
> to be used.
> Drew, could you provide a patch for the myri10ge driver to show how the
> "receive in page" interface works?

Hi,

Thank you for the many fixes, and especially for the counters!

I was working on testing the myri10ge patch, and I ran into a few
problems.  I've attached a patch to inet_lro.c to fix some of them,
and a patch to myri10ge.c to show how to use the page based
interface.  Both patches are signed off by Andrew Gallatin
<gallatin at myri.com>

First, the LRO_MAX_PG_HLEN is still a problem.  Minimally sized 60
byte frames still cause problems in lro_gen_skb due to skb->len
going negative.  Fixed in the attached patch.  It may be simpler
to just drop LRO_MAX_PG_HLEN to ETH_ZLEN, but I'm not sure if
that is enough.  Are there "smart" NICs which might chop off padding
themselves?

Second, you still need to set skb->ip_summed = CHECKSUM_UNNECESSARY
when modified packets are flushed, else the stack will see bad
checksums for packets from CHECKSUM_COMPLETE drivers using the
skb interface.  Fixed in the attached patch.

Third, for some reason I have yet to figure out, this version of the
patch takes a while to "ramp up", but only when the receiver MTU
is 9000 and the sender MTU is 1500.  If the receiver MTU is also
1500, even a 10 second netperf easily shows line rate.  I don't
see this with our LRO, and I'm so far at a loss to explain it.

Here is the first 3 seconds of  output from a simple program
which diffs the interface counters once / sec.
        Ipkts       IBytes        Opkts       Obytes

rx MTU=9000:
            0            0            0            0
           72        99092           14         1102
          105       158970            7          420
       750324   1135987084        17804      1068240
       814427   1233042478        18529      1111740

<....>

rx MTU=1500
            0            0            0            0
       441344    668180168        13132       788182
       815938   1235328656        18716      1122960
       817698   1237994772        18628      1117680
<.....>

Once it has ramped up, the bandwidth is fine, and there
doesn't seem to be much difference between setting the
receiver MTU to 1500 or 9000.  But it takes a very long
netperf run to overcome the ramp up period.

Fourth, I did some traffic sniffing to try to figure out what's going
on above, and saw tcpdump complain about bad checksums.  Have you tried
running tcpdump -s 65535 -vvv?  Have you also seen bad checksums?
I seem to see this for both page- and skb-based versions of the driver.

Last, the pointer to the TCP header in __lro_proc_segment() in the
unaccelerated vlan hdr case needs to also be offset by vlan_hdr_len
from skb->data.   I've fixed this in the attached patch.



Thanks,

Drew
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: inet_lro.diff
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20070730/fe20baff/attachment.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: myri10ge_lro.diff
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20070730/fe20baff/attachment.txt>