netdev - Re: the side effect of using copy skb instead of skb

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <87y4w4jprx.fsf@nemi.mork.no>
Date:	Tue, 08 Jul 2014 09:03:30 +0200
From:	Bjørn Mork <bjorn@...k.no>
To:	joey ming <joey.zming@...il.com>
Cc:	jim_baxter@...tor.com, alexey.orishko@...ricsson.com,
	netdev@...r.kernel.org, zhao.ming9@....com.cn
Subject: Re: the side effect of using copy skb instead of skb_clone in cdc ncm/mbim driver

joey ming <joey.zming@...il.com> writes:

> Hi, I was hoping you can help me with some questions.
>
> some qualcomm LTE modem has two mode: ECM and QC_NCM.

Yes, and worse than that: You can even mix and match these on UL and DL.

> QC_NCM is the
> protocol which qualcomm defined,the  difference between qc-ncm and cdc-mbim
> are:
> data pipe:     NDP sig vary
>                    qc-ncm  packets are composed of  NTH+datagram+NTB  but
> mbim/ncm are NTH+NTB+datagram

Both these schemes are allowed in NCM (and therefore MBIM). There is no
required datagram nd NTB order. Proper alignment is the only requirement.
So the only real difference is the signature, which AFAIK is
configurable on the QC_NCM devices.

> control pipe: the biggest difference is control pipe. QC-NCM uses QMI cmd
> to control device but mbim/ncm uses CDC cmd .

Yes, but this usage is actually also within the NCM spec.  The only
issues making a typical QC_NCM capable device fail as a CDC NCM class
device are
 - the default signature
 - the USB descriptors

Both are of course configurable by the modem vendors using these
chips/firmware toolkits, so it's possible to create standard CDC NCM
class devices out of these.  But you probably don't want to do that,
because there aren't any NCM class drivers which will let you access the
QMI control channel (except the one you have made :-), and you need QMI
for modem management.

> so for a QC-NCM driver, we can borrow the code of data pipe from
> cdc-mbim/ncm. for a real LTE modem(DL speed:150Mbps) the qc-ncm driver
> speed which used skb_clone in qc_ncm_rxfixup  can reach 126Mbps,but at the
> same time and same envirmont ecm driver can reach 144Mbps for the same
> modem.
> Thanks for google, I found  Bjorn Mork's discussion about buffering
> restrictions of cdc-ncm host driver.
>  the qc-ncm driver speed can reach 145Mbps  if  I used copy skb instead of
> skb_clone.
> as we know, the aggregation protocol(such as cdc-ncm,cdc-mbim,qc-ncm )
> major purpose are improving performance of CPU by reducing interrupts.
> Compared ecm and qc-ncm test result, I thought we didn't reach the goal.
> the interrupts decreased 20% BUT the driver load was no change.
> I used Intel Core Duo laptop and oprofiled in 3.12 kernel:
>   QC-NCM           0.3146%(CPU0)    0.2899%(CPU1)  vmlinux
>  vmlinux                  memcpy
>                           0.2198%(CPU0)    0.1864%(CPU1)  usbnet
>         usbnet                   usbnet
>                           0.0423 %(CPU0)   0.0385%(CPU1)  qmi_wwan
>      qmi_wwan              /qmi_wwan
>
>   CDC-ECM         0.2470%(CPU0)     0.1735%(CPU1)  usbnet
> usbnet                   /usbnet
>                           0.1341%(CPU0)     0.0132%(CPU1)  vmlinux
>          vmlinux                  memcpy
>                           0.0335%(CPU0)      0.0387%(CPU1)  qc_ncm
>         qc_ncm                  /qc_ncm
>
> for a high speed modem(DL >100Mbps), Is it a appropriate solution use copy
> skb? as above test result, memcpy load was higher than usbnet.

copy is the only solution.  With aggregation on the USB link we can
choose either
 - USB buffers which are much bigger than one IP packet, or
 - splitting some IP packets over two USB buffers

Either way we'll have to copy the IP packets before handing them over to
the upper layers.   Cloning oversized skbs is not a good idea.

The problem you found is that there is no real value in this USB layer
aggregation on Linux.  It's just pointless complexity. A plain ECM
driver transmitting ethernet frames back to back will achieve the exact
same utilisation of the USB link, without all the aggregation complexity
and overhead.

I don't know why the USB-IF found it necessary to create these
aggregation protocols, but I assume there are other OS's than Linux out
there (running on either hosts or devices) which are unable to use the
full USB bandwith with ECM.

But the results you show above is the primary reason why I chose to
completely ignore the QC_NCM bastard protocol.  AFAIK all devices
capable of using this protocol can also do the plain ECM, and will do so
by default.  So they are supported by the qmi_wwan driver with no more
fuzz, doing the maximum speed the devices are capable of.

Thanks a lot for testing this and providing the results!


Bjørn
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html