DPAA: Software Portal selection for network interface

[Madalin-cristian Bucur]

Hi Sebastian,

> -----Original Message-----
> From: Sebastian Huber [mailto:sebastian.huber at embedded-brains.de]
> Sent: Thursday, June 22, 2017 3:42 PM
> To: linuxppc-dev at lists.ozlabs.org
> Cc: Madalin-cristian Bucur <madalin.bucur at nxp.com>
> Subject: DPAA: Software Portal selection for network interface
> 
> Hello,
> 
> as far as I understand the software portal selection for a network
> interface is done in
> 
> static int dpaa_eth_probe(struct platform_device *pdev)
> {
>      [...]
>      channel = dpaa_get_channel();
>      if (channel < 0) {
>          dev_err(dev, "dpaa_get_channel() failed\n");
>          err = channel;
>          goto get_channel_failed;
>      }
> 
>      priv->channel = (u16)channel;
> 
>      /* Start a thread that will walk the CPUs with affine portals
>       * and add this pool channel to each's dequeue mask.
>       */
>      dpaa_eth_add_channel(priv->channel);
> 
> with
> 
> static int dpaa_get_channel(void)
> {
>      spin_lock(&rx_pool_channel_init);
>      if (!rx_pool_channel) {
>          u32 pool;
>          int ret;
> 
>          ret = qman_alloc_pool(&pool);
> 
>          if (!ret)
>              rx_pool_channel = pool;
>      }
>      spin_unlock(&rx_pool_channel_init);
>      if (!rx_pool_channel)
>          return -ENOMEM;
>      return rx_pool_channel;
> }
> 
> which always returns the same pool channel (e.g. 0x401) if successful.

That's correct.

> This means all the QMan portal_isr() are distributed round-robin to all
> affine portals. Is there some way to configure the software portal for a
> specific network interface, e.g. use processors 0, 1, 2, 3 for one
> interface,and 4, 5, 6, 7 for another?

We've stripped all configurability and advanced features from the initial
DPAA submission. We don't have options to configure this. On the other hand,
the ingress queues are held active in the portal, resulting in one CPU doing
dequeues while there are frames available. This is done to avoid frame 
reordering, improving termination performance (and forwarded TCP performance).

The downside is that we're not saturating all CPUs with traffic. We're
currently working on adding Rx hashing, to be able to maintain intra-flow
frame order while spreading processing on all CPUs.

Meanwhile, would RPS (Receive Packet Steering) be a solution for you?

> --
> Sebastian Huber, embedded brains GmbH
> 
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax     : +49 89 189 47 41-09
> E-Mail  : sebastian.huber at embedded-brains.de
> PGP     : Public key available on request.
> 
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.


Regards,
Madalin