netdev - Re: UDP sockets oddities

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <354e6c3a-1771-e8a7-24dd-1b70266563af@gmail.com>
Date:   Fri, 25 Aug 2017 20:25:26 -0700
From:   Florian Fainelli <f.fainelli@...il.com>
To:     Eric Dumazet <eric.dumazet@...il.com>
Cc:     netdev@...r.kernel.org, pabeni@...hat.com, willemb@...gle.com,
        davem@...emloft.net
Subject: Re: UDP sockets oddities



On 08/25/2017 06:52 PM, Eric Dumazet wrote:
> On Fri, 2017-08-25 at 18:17 -0700, Florian Fainelli wrote:
>> On 08/25/2017 04:57 PM, Eric Dumazet wrote:
>>> On Fri, 2017-08-25 at 16:18 -0700, Florian Fainelli wrote:
>>>
>>>> Eric, are there areas of the stack where we are allowed to drop packets,
>>>> not propagate that back to write(2) and also not increment any counter
>>>> either, or maybe I am not looking where I should...
>>>
>>> What happens if you increase these sysctls ?
>>
>> I don't see packet loss after I tweak these two sysctls according to
>> your suggestions.
>>
>> Tweaking eth0's sysctls did not change anything, but tweaking gphy's
>> sysctl resolved the loss. This was a little surprising considering that
>> gphy is an IFF_NO_QUEUE interface and eth0 is the conduit interface that
>> does the real transmission.
>>
>> Does that make sense with respect to what I reported earlier? Should I
>> try to dump the neigh stats?
> 
> Note that if you had TCP traffic, the neighbour would be constantly
> confirmed and no losses would happen.

OK, that still sounds like quite a lot for a not so long UDP session (60
seconds).

I was finally able to get a better capture by switching to an ARM64
kernel, and as confirmed this is all coming from the neighbour code:

# Event count (approx.): 1970
#
# Children      Self  Trace output

# ........  ........
....................................................................
#
     3.10%     3.10%  skbaddr=0xffffffc2fa22a800 protocol=2048
location=0xffffff80086e53f4
            |
            ---write
               el0_svc_naked
               sys_write
               vfs_write
               __vfs_write
               sock_write_iter
               sock_sendmsg
               inet_sendmsg
               udp_sendmsg
               udp_send_skb
               ip_send_skb
               ip_local_out
               ip_output
               ip_finish_output
               ip_finish_output2
               neigh_resolve_output
               __neigh_event_send
               kfree_skb
               kfree_skb

     3.10%     3.10%  skbaddr=0xffffffc2fa22a900 protocol=2048
location=0xffffff80086e53f4
            |
            ---write
               el0_svc_naked
               sys_write
               vfs_write
               __vfs_write
               sock_write_iter
               sock_sendmsg
               inet_sendmsg
               udp_sendmsg
               udp_send_skb
               ip_send_skb
               ip_local_out
               ip_output
               ip_finish_output
               ip_finish_output2
               neigh_resolve_output
               __neigh_event_send
               kfree_skb
               kfree_skb

     3.10%     3.10%  skbaddr=0xffffffc2fa22aa00 protocol=2048
location=0xffffff80086e53f4
            |
            ---write
               el0_svc_naked
               sys_write
               vfs_write
               __vfs_write
               sock_write_iter
               sock_sendmsg
               inet_sendmsg
               udp_sendmsg
               udp_send_skb
               ip_send_skb
               ip_local_out
               ip_output
               ip_finish_output
               ip_finish_output2
               neigh_resolve_output
               __neigh_event_send
               kfree_skb
               kfree_skb

> 
> I guess we should an SNMP counter for packets dropped in neigh queues.

It would. Since the call trace involves udp_send_skb() how come we are
not returning an error to write(2)? are there other code paths where the
neighbor code can do drops like these?
-- 
Florian