netdev - Re: BUG:af_packet fails to TX TSO frames

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAF=yD-+YUZdCCLXdQEG5AA0Pkrf-OrWGwySWf7mqH5BhO-Vr+g@mail.gmail.com>
Date:   Wed, 11 Oct 2017 20:19:56 -0400
From:   Willem de Bruijn <willemdebruijn.kernel@...il.com>
To:     Anton Ivanov <anton.ivanov@...bridgegreys.com>
Cc:     Anton Ivanov <anton.ivanov@...-begemot.co.uk>,
        Network Development <netdev@...r.kernel.org>,
        David Miller <davem@...emloft.net>
Subject: Re: BUG:af_packet fails to TX TSO frames

On Wed, Oct 11, 2017 at 6:01 PM, Anton Ivanov
<anton.ivanov@...bridgegreys.com> wrote:
> [snip]
>
>> This will be tomorrow though, it is late here.
>>
>> The only obvious difference I can see at this point is that I am using
>> iovs and sending the vnet header as iov[0] and the data in pieces after
>> that while your code is doing a send() for the whole frame. This should
>> not make any difference though - it all ends up as an iov internally in
>> the kernel.
>
> Spoke too soon. It is not reporting any errors, but there is nothing
> coming out on the actual Ethernet.

It works for me on various platforms. On the receiver, drop these fake
tcp packets in iptables and read them with tcpdump

   iptables -A PREROUTING -t raw -p tcp --dport 9 -j DROP
   tcpdump src $src_ip

Note that not all combinations of flags are supported by the kernel
and that some flags have non-obvious behavior (disable a feature, in
place of enable it).

Specifically, mtu sized packets either must not pass a vnet_hdr or
must pass one with gso explicitly disabled ('-G').

  psock_txring_vnet -s $src_ip $dst_ip -l 1400
  psock_txring_vnet -s $src_ip $dst_ip -l 1400 -v -G
  psock_txring_vnet -s $src_ip $dst_ip -l 1400 -N
  psock_txring_vnet -s $src_ip $dst_ip -l 1400 -N -v -G

Conversely, packets that exceed mtu have to have the gso flags in the
virtio_net_hdr:

  psock_txring_vnet -s $src_ip $dst_ip -l 4400 -v
  psock_txring_vnet -s $src_ip $dst_ip -l 4400 -N -v

When sending a large packet, but not passing a virtio_net_hdr along
('-v'), the test fails with

  psock_txring_vnet: send: Message too long

When passing a header along, but not disabling gso, the packet is
indeed dropped silently.

I verified correct segmentation with three modes of ethtool

  ethtool -K eth0 tso off gso off
  ethtool -K eth0 tso off gso on
  ethtool -K eth0 tso on gso on

by reading tcpdump on the sender.

The receive side results are the same with dev_queue_xmit and
packet_direct_xmit ('-q') mode. With direct_xmit, the packets are not
observed on the send side.