netdev - Re: [BUG] Kernel recieves DNS reply, but doesn't deliver it to a waiting application

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20130116203644.8eb2a16f.bircoph@gmail.com>
Date:	Wed, 16 Jan 2013 20:36:44 +0400
From:	Andrew Savchenko <bircoph@...il.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	netdev@...r.kernel.org
Subject: Re: [BUG] Kernel recieves DNS reply, but doesn't deliver it to a
 waiting application

Hello,

On Fri, 28 Dec 2012 10:11:03 -0800 Eric Dumazet wrote:
> On Sun, 2012-12-23 at 15:06 +0400, Andrew Savchenko wrote:
[...]
> > I hit this bug again on uptime 11 days on 3.7.0 vanilla kernel.
> > See kernel config, /prot/net/upd, netstat -s and dropwatch logs
> > attached to this mail. This bug happens on UDP DNS requests only,
> > TCP requests work fine, see dig.log attached.
> > 
> > Increasing of net.ipv4.udp_mem from
> > 24150        32201   48300
> > to
> > 100000       150000  200000
> > helps, but I'm afraid only temporary again.
> > 
> > Dropwatch data was collected in the following way:
> > - dropwatch.bug.* files contain data obtained after bug occurred;
> > - dropwatch.*.background files contain background data when no
> >   host or dig test was running; this system has active firewall
> >   and complicated routing, ipv6 disabled via sysctl, etc, so some
> >   drops are normal;
> > - dropwatch.*.host.request shows dropped packets recorded during
> >   host ya.ru request; of course, during this time some background
> >   packets were recorded as well (dropwatch doesn't support filtering
> >   at this moment);
> > - dropwatch.nobug.* data was collected after the bug was
> >   workarounded via net.ipv4.upd_mem as described above.
> > 
> > As can be seen from dropwatch logs, drop in __udp_queue_rcv_skb+61
> > happens only on host request on bug conditions, thus something is
> > wrong there.
> > 
> > Best regards,
> > Andrew Savchenko
> 
> Thanks a lot !
> 
> I see strange drops in dev_hard_start_xmit()
> 
> l2tp needs some care.
> 
> Please try the following patch, as skb_cow_head() API
> doesnt really ease skb->truesize exact tracking anyway, better not mess
> with it. 

Sorry for the delay, but I was able to reboot kernel only today.
Your patch is applied on top of the 3.7.2 vanilla kernel.

l2tp works fine and /proc/net/udp tx_queue values are normal now, see
attached /prot/net/udp output. This is a good hint that problem is
probably solved, but we need to wait at least several weeks to be
sure.

Best regards,
Andrew Savchenko

Download attachment "proc.net.udp" of type "application/octet-stream" (4224 bytes)

Content of type "application/pgp-signature" skipped