netdev - Re: [PATCH net] ipv4: fix race in concurrent ip_route_input

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 20 Nov 2013 15:29:18 -0500 (EST)
From:	David Miller <davem@...emloft.net>
To:	ast@...mgrid.com
Cc:	edumazet@...gle.com, netdev@...r.kernel.org
Subject: Re: [PATCH net] ipv4: fix race in concurrent ip_route_input_slow()

From: Alexei Starovoitov <ast@...mgrid.com>
Date: Tue, 19 Nov 2013 19:12:34 -0800

> CPUs can ask for local route via ip_route_input_noref() concurrently.
> if nh_rth_input is not cached yet, CPUs will proceed to allocate
> equivalent DSTs on 'lo' and then will try to cache them in nh_rth_input
> via rt_cache_route()
> Most of the time they succeed, but on occasion the following two lines:
> 	orig = *p;
> 	prev = cmpxchg(p, orig, rt);
> in rt_cache_route() do race and one of the cpus fails to complete cmpxchg.
> But ip_route_input_slow() doesn't check the return code of rt_cache_route(),
> so dst is leaking. dst_destroy() is never called and 'lo' device
> refcnt doesn't go to zero, which can be seen in the logs as:
> 	unregister_netdevice: waiting for lo to become free. Usage count = 1
> Adding mdelay() between above two lines makes it easily reproducible.
> Fix it similar to nh_pcpu_rth_output case.
> 
> Fixes: d2d68ba9fe8b ("ipv4: Cache input routes in fib_info nexthops.")
> Signed-off-by: Alexei Starovoitov <ast@...mgrid.com>
> ---
> 
> David,
> 
> looks like caacf05e5ad1 ("ipv4: Properly purge netdev references on uncached routes.")
> fixed the race for nexthop/rth_output, but missed it for rth_input.
> I'm not sure what was the assumption why it's not needed there.
> We're definitely seeing it every 12-24hr during nightly tests.
> There are several bugs on ubuntu and debian forums with similar description.
> Some were closed, since folks struggled to reproduce it.
> It took us more than a month to debug it.
> Please queue for stable.

Your analysis is accurate and your fix is absolutely correct, applied
and queued up for -stable, thanks!
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html