[PATCH 0/4][RFC] lro: Generic Large Receive Offload for TCP traffic

Andrew Gallatin gallatin at myri.com
Tue Jul 31 06:32:09 EST 2007

Jan-Bernd Themann wrote:
> Hi,
> this patch set contains the latest generic LRO code, a Kconfig / Makefile
> and an eHEA patch demonstrating how the "aggregate SKB" interface has to
> to be used.
> Drew, could you provide a patch for the myri10ge driver to show how the
> "receive in page" interface works?


Thank you for the many fixes, and especially for the counters!

I was working on testing the myri10ge patch, and I ran into a few
problems.  I've attached a patch to inet_lro.c to fix some of them,
and a patch to myri10ge.c to show how to use the page based
interface.  Both patches are signed off by Andrew Gallatin
<gallatin at myri.com>

First, the LRO_MAX_PG_HLEN is still a problem.  Minimally sized 60
byte frames still cause problems in lro_gen_skb due to skb->len
going negative.  Fixed in the attached patch.  It may be simpler
to just drop LRO_MAX_PG_HLEN to ETH_ZLEN, but I'm not sure if
that is enough.  Are there "smart" NICs which might chop off padding

Second, you still need to set skb->ip_summed = CHECKSUM_UNNECESSARY
when modified packets are flushed, else the stack will see bad
checksums for packets from CHECKSUM_COMPLETE drivers using the
skb interface.  Fixed in the attached patch.

Third, for some reason I have yet to figure out, this version of the
patch takes a while to "ramp up", but only when the receiver MTU
is 9000 and the sender MTU is 1500.  If the receiver MTU is also
1500, even a 10 second netperf easily shows line rate.  I don't
see this with our LRO, and I'm so far at a loss to explain it.

Here is the first 3 seconds of  output from a simple program
which diffs the interface counters once / sec.
        Ipkts       IBytes        Opkts       Obytes

rx MTU=9000:
            0            0            0            0
           72        99092           14         1102
          105       158970            7          420
       750324   1135987084        17804      1068240
       814427   1233042478        18529      1111740


rx MTU=1500
            0            0            0            0
       441344    668180168        13132       788182
       815938   1235328656        18716      1122960
       817698   1237994772        18628      1117680

Once it has ramped up, the bandwidth is fine, and there
doesn't seem to be much difference between setting the
receiver MTU to 1500 or 9000.  But it takes a very long
netperf run to overcome the ramp up period.

Fourth, I did some traffic sniffing to try to figure out what's going
on above, and saw tcpdump complain about bad checksums.  Have you tried
running tcpdump -s 65535 -vvv?  Have you also seen bad checksums?
I seem to see this for both page- and skb-based versions of the driver.

Last, the pointer to the TCP header in __lro_proc_segment() in the
unaccelerated vlan hdr case needs to also be offset by vlan_hdr_len
from skb->data.   I've fixed this in the attached patch.


-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: inet_lro.diff
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20070730/fe20baff/attachment.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: myri10ge_lro.diff
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20070730/fe20baff/attachment.txt>

More information about the Linuxppc-dev mailing list