netdev - Re: [PATCH net-next] ip: avoid OOM kills with large UDP sends over loopback

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210622095725.53fbf399@kicinski-fedora-PC1C0HJN.hsd1.ca.comcast.net>
Date:   Tue, 22 Jun 2021 09:57:25 -0700
From:   Jakub Kicinski <kuba@...nel.org>
To:     Paolo Abeni <pabeni@...hat.com>
Cc:     davem@...emloft.net, netdev@...r.kernel.org, willemb@...gle.com,
        eric.dumazet@...il.com, dsahern@...il.com, yoshfuji@...ux-ipv6.org,
        Dave Jones <dsj@...com>
Subject: Re: [PATCH net-next] ip: avoid OOM kills with large UDP sends over
 loopback

On Tue, 22 Jun 2021 12:07:27 +0200 Paolo Abeni wrote:
> On Mon, 2021-06-21 at 16:13 -0700, Jakub Kicinski wrote:
> > Dave observed number of machines hitting OOM on the UDP send
> > path. The workload seems to be sending large UDP packets over
> > loopback. Since loopback has MTU of 64k kernel will try to
> > allocate an skb with up to 64k of head space. This has a good
> > chance of failing under memory pressure. What's worse if
> > the message length is <32k the allocation may trigger an
> > OOM killer.  
> 
> Out of sheer curiosity, are there a large number of UDP sockets in such
> workload? did you increase rmem_default/rmem_max? If so, could tuning
> udp_mem help?

It's a handful of sockets, < 10.

> > diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
> > index c3efc7d658f6..a300c2c65d57 100644
> > --- a/net/ipv4/ip_output.c
> > +++ b/net/ipv4/ip_output.c
> > @@ -1095,9 +1095,24 @@ static int __ip_append_data(struct sock *sk,
> >  				alloclen += rt->dst.trailer_len;
> >  
> >  			if (transhdrlen) {
> > -				skb = sock_alloc_send_skb(sk,
> > -						alloclen + hh_len + 15,
> > +				size_t header_len = alloclen + hh_len + 15;
> > +				gfp_t sk_allocation;
> > +
> > +				if (header_len > PAGE_SIZE)
> > +					sk_allocation_push(sk, __GFP_NORETRY,
> > +							   &sk_allocation);  
> 
> Could an additional __GFP_NOWARN be relevant here?

We always set GFP_NOWARN for heads thru kmalloc_reserve().